An artifact is a noticeable difference between an uncompressed signal and a lossily-compressed copy. Lossy encoding can result in very different kind of artifacts/distortions. Sometimes it's not easy to define why the encoding is non-transparent. There are however many typical encoding artifacts.

ff123's Audio Artifact Training Page

In psychoacoustics, transparency is the ideal result of lossy data compression. If a lossily compressed result is perceptually indistinguishible from the uncompressed input, then the compression can be declared to be transparent. In other words, transparency is the situation where artifacts are nonexistant or imperceptible.

ABX is a method for determining by listening whether two wav files are audibly different from each other. The method is most useful for listening to potential differences near the threshold of audibility. A key feature of this method is that the tests are performed "blind," or without the listener's knowledge of what the file-under-test is. Another key feature is that the influence of chance on the results can be reduced by performing multiple tests (trials).