Its probably your heap of rubbish at fault. In almost every case so far investigated, the cause of unstable reception has been frame slips. This is a system configuration problem, and nothing to do with spandsp itself. Modems cannot tolerate even single samples removed or inserted in the audio stream. A few investigated reports have related to modem signals which are out of spec. Over a long period spandsp has gradually been tuned to be more tolerant of these problems. There should be very failures of this type with the current spandsp.
You may be faxing over IP, or through a direct PSTN interface, but each has issues when using modems of any kind.
FAXing over an IP channel is best handled with a proper FAX over IP (FoIP) protocol, rather than using VoIP protocols. T.37 and T.38 are the main FoIP protocols. They convert the modem tones to binary control and image data at the PSTN interfaces, and send that data across the IP path. This is solid, reliable, and bandwidth efficient. However, not all VoIP equipment supports FoIP properly, and many people try to use VoIP protocols to carry FAXes. This note explains some of the key reasons this is not always successful.
Of the common codecs used for VoIP, only A-law and u-law are capable of passing FAX data successfully. Low bit rate codecs will not preserve the signal well enough for the fast modems used for FAX images. FAX transmission generally consists of burst of a slow V.21 modem, sending control messages at 300bps, and a fast modem (usually V.17, V.27ter or V.29) sending the image data at 2400bps to 14400bps. Some low bit rates codecs might just about pass the 300bps FAX control messages OK. None will pass the image data successfully.
If you are using A-law or u-law, there are other factors which can cause anything from slightly erratic operation to no operation at all. Many systems have synchronisation problems, which cause frame slips.
Frame slips are a general problem with PSTN communications when the transmitting and receiving clock are not synchronised. In a properly configured system they should be synchronised, but in a large number of installations they are not. When the clocks drift far enough apart that they are a whole audio sample out of step, the receiver jumps one sample, and either an extra sample appears in the audio data stream (if the receive clock is too fast), or one is missing (if the receive clock is too slow). For voice calls each frame slip is heard as a sharp click. A remarkable number of systems have this problem, and people just live with it. FAXing cannot. Each time a frame slip occurs, the receiving FAX modem will loose sync with the transmitter. FAX modems are designed to work on very bad noisy lines. They do not tolerate hiccups in time. This problem is fixable. You can be certain the entire PSTN will never synchronise itself to your machine. You can ensure you are synchronised to the PSTN.
If you have frame slips, but the slip rate is not too high, your FAXes might get through. FAX machines can retry bad pages. If frame slip occur only once each minute or two, retries will probably get the pages through. However, such haphazard behaviour is hardly a good solution.
So, you are using a-law or u-law, and your trunks are properly sychronised. What else can go wrong? Timing issues in the IP path are the main thing. The two machines at the ends of the IP path generally have no means to accurately synchronise their clocks. Their first priority may be to synchronise to the PSTN links attached to them. Surprisingly, this may be a magic cure for timing issues. Public exchanges use rhubidium atomic clocks for timing, so any two exchanges across the global should be very close to in-sync. If two VoIP machines are synching to the PSTN, they should gain the same clock accuracy benefit.
The VoIP packets typically contain 20ms of voice data. As the timing gradually drifts between the two ends, some trickery is required to deal with it, while avoiding a hop as big as 20ms. Systems vary in the strategy they use, but there will be some kind of jump in the audio stream. If the two ends are digitally (ISDN, SS7, etc) connected to their local PSTN networks, the drift should be extremely small. Public exchanges use rhubidium atomic clocks for timing, so any two exchanges across the global should be very close to in-sync. If there is no digital PSTN connection, PBXs and especially free standing computer nodes have very poor timing, and the drift could be quite fast. Each time the data is fudged to overcome these timing issues, the modem will loose sync.