No sane person should be trying to send FAXes over VoIP channels. However, in development work we do some insane things as part of our system investigations.
In developing T.38 for free telephony platforms, such as Asterisk, I have begun finding some true nastiness in the behaviour of real world ATAs. I have been working with a couple of ATAs using the Myson-Century CS6220 chipset, which used to be the Netergy T2U. A number of ATAs from Taiwan seem to use this, and they all use the silicon vendor's software. That means the only difference in their behaviour is the difference between software revisions. These are NOT crude, low end boxes. They are sold at fairly high prices as deluxe products.
The following pictures show the audio received from a FAX machine connected to a Planet VIP-160 ATA. The ATA was set to use u-law audio. These pictures are of audio received within Asterisk, after decoding the RTP stream. Communication was across a lightly loaded LAN, so there was no packet loss, and stationary jitter. The first picture was produced with echo cancellation switched on in the VIP-160's configuration menus. The first block of sound is the 2100Hz answer tone. After a tiny gap comes the first message. A spandsp based soft-fax within Asterisk then responded, which happened during the long gap. Then the ATA answered back Note the way the audio ramps linearly during this response. The timescale is not shown here, but that ramp takes about one second. Other tests seem to indicate the audio ramps this way whenever a burst of transmission immediately follows a burst of reception.
The second picture shows a similar test, but this time the echo canceller in the VIP-160 was turned off in the configuration menu. Notice the echo from the signal transmitted by spandsp is now visible in the gaps. Note also that the audio from the ATA no longer ramps. It cuts in cleanly. I tried this test a couple of time, and that nasty ramping is definitely a function of having EC switched on. It looks like this ATA is not performing real echo cancellation at all, which is what its spec says it should. It looks like it is performing a crude echo suppression, rather like an old crude analogue line speakerphone. It cuts the amplitude of the echo, but has a really bad effect on the good signal too.
With EC switched on, FAXing over ulaw does not work at all. Decoding the ramping audio signal is too hit and miss. With EC switched off, the FAX machine can receive a couple of pages, but still fails. I haven't found the cause of that yet.
The T.38 spec is very open ended, and implementations can vary hugely without contravening the spec. One interesting area I have found, with some bandwidth implications, is the initial T.38 interaction. I have been calling a T.38 capable ATA, specifying both u-law and T.38 usage in the SIP setup. I need to specify both, as I don't know if the box is T.38 capable. FAX over u-law isn't reliable, but FAX over T.38 to a box that does not understand it means a certain failure. I expected only one kind of response. However, the box sends back both u-law audio in an RTP stream, and T.38 in a UDPTL stream. Once T.38 responses are received by the ATA, it seems to stop the audio stream. However, up to that point the use of T.38 has actually increased the bandwidth demand. This could have unexpected implications for some people. Especially since the audio is always u-law or a-law.
This is dumb. The ATA has been told in the SIP setup that the other party is T.38 capable.