I'm actually using cgminer 3.1.1 anyway. Not for any reason other than that's what was current when I cloned it. I wonder if it's a good idea to update everything to the current version. I've edited many files that contain driver related definitions so I'd have to do some sort of merge if I want to get a new version. I should set it up as a proper fork anyway but just have avoided spending time on that.
With usbread in this version if you give it a read count it will timeout if less bytes are available/read. I'm ignoring timeouts for polling reply data because some replies are of different lengths. If I move to a fixed 16 byte record length even when less is needed then every usbread is for the fixed length and it always returns immediately when data is pending, and presumably if two packets come in succession they won't pile up as they do now. What I'd ideally like is a callback function for each packet received with no polling. For now it's a second thread that just repeatedly polls with usbread for any replies and queues them for other threads to use.
Anyway, watching data with usbmon I see that sometimes for no visible (logged) reason it will just decide to do a control sequence to re-init the device. I don't know why it's doing that. It seems to happen after a nonce is read and a -84 code where normally there is a -115 code logged. But so far I haven't found a reference to what those codes mean. Whatever -84 means seems to cause a device re-init, but as doesn't appear to be related to a bad nonce value. Sometimes it happens even when an accepted good nonce is found.
Mapping of stream data onto a USB bulk pipe can be kind of awkward. Bulk pipes send packets of a fixed size (the descriptors document the size of the maximum size).
USB is host driven, so the communication with the device is always polled (on the wire at least) by the host. Effectively the device asks the device "do you have any data for this pipe?". The device then sends data. This can be anything up to the maximum packet size and there is no negotiation between the host and the device on how much the device is expecting.
If the device wants to send more data than the host is expecting, then it results in a babble error and this can ultimately end up in a device reset.
Reads that are larger than the maximum packet size for that pipe will be split into multiple packets of the maximum packet size with the last packet being equal to or less than the maximum packet size. (ie 150 byte read = 64 byte read + 64 byte read + 22 byte read).
This packet framing can be a problem with serial devices because you often don't know how much data is pending on the device side, or you're trying to read a subset of the data pending on the device side (because you're trying to read just enough data that you need for now).
For these serial devices, you should always read a multiple of the maximum packet size. If a device sends less, this is called a short read. Usually this is fine and how the protocol expects devices to operate (but this can be treated as an error too).
One last note about reads that are a multiple of the packet size. In that case, an extra 0-byte read will need to happen to ensure the proper framing occurs (ie data has 192 bytes to send, the host should read 256 bytes, which end up on the wire as 4 64-byte reads, with the last read being a 0-byte short read).
The kernel USB serial driver handles all of this framing for you, but it's certainly possible to handle this using libusb as well.
I'd suggest just always doing a large read (4096 bytes?) that is always a multiple of the device packet size and putting the data you read into an internal buffer. Then pull data off that buffer as you need it.
Also, libusb does support asynchronous transfers. You can place a read on a bulk pipe and you can be notified when the read finishes. The API could probably be easier to use in this case.