The ubiquitin-like protein ISG15 is activated in response to type 1 interferons and its conjugation to proteins regulates the response to bacterial and viral infection. Its subsequent deconjugation, by a single human enzyme, USP18, critically controls interferon signaling and the defense against pathogens. Accordingly, human mutations in USP18 give rise to interferonopathies. However, the molecular determinants underlying USP18 specificity for ISG15 remain elusive. We have identified key mechanisms that underlie the specificity of USP18 towards ISG15, by taking advantage of USP18’s largely unstudied human paralog USP41. Remarkably, we show that USP41 completely lacks the ability to perform deISGylation, despite possessing a catalytic domain that is 97% identical to that of USP18. Using a variety of in vitro and in vivo assays, we performed a comparative analysis to reveal new mechanisms of deISGylation. We define a short, conserved stretch of amino acids, that we term the TAIL motif, as well as a conserved leucine residue in the catalytic domain, which are necessary for deISGylation. Structural analysis revealed that these residues contact ISG15 and are conserved between USP18 and PLpro from SARS-Cov2, underlining their importance in ISG15 specification. Finally, while little is known about USP41, we found that it can bind and negatively regulate FAT10, a poorly studied Ubl which is also involved in the interferon response. While a FAT10 deconjugating enzyme has long been postulated, no such enzyme has ever been described.