More recently, biometric systems have spread for modern security applications. Unfortunately, these systems have experienced several attempts of hacking. If biometric databases are compromised and stolen, biometrics in these databases will be lost forever. Consequently, there is an immediate need to introduce new upgradable biometric systems. The concept behind cancelable biometrics is to convert biometric data to alternative templates, which cannot be easily used by the impostor or intruder, and can be eliminated if breached. In this paper, the inverse filter is utilized in a cancelable face recognition system. In this system, masked biometric images are generated by blurring, noise addition and then inverse filtering. It is well-known in the image processing theory that inverse filtering leads to noise enhancement, which is an undesired effect in image restoration. In contrary, this effect will be desired in cancelable biometric systems. If the noise is magnified with an appropriate extent, it can mask the original biometrics leading to cancelable templates. This is the theory behind the proposed system. The proposed system is applied on the Olivetti and Oracle (ORL) dataset. Simulation results using evaluation metrics such as non-invertibility, unlinkability, visual inspection, False Positive Rate (FPR), False Negative Rate (FNR), Equal Error Rate (EER), Decidability, correlation coefficient, Area under the Receiver Operating Characteristic (AROC) curve demonstrate that the proposed system is resistant to intruders and hackers. Hence, it is efficient for several security applications.