One Ringgit and five Ringgit Malaysian banknotes reader using backlight mechanism and image processing techniques

1. Introduction

Banknote readers are machines that are used to check whether the received banknotes are genuine or fake. These devices can be found in a variety of automated equipment, including supermarket self-check-out machines, laundromat washing machines, parking ticket paying machines, automatic fare collecting machines, public transportation ticket selling machines, and vending machines. The operating procedures for these machines’ banknotes reading devices entail inspecting the banknotes that have been entered into the machine and running a series of tests to see if they are counterfeit or not. These currency acceptors must be accurately configured for each item to be accepted since the specifications for each banknote are different.

Generally, the banknote reader suitable for Malaysian banknotes can be categorized into four processes: FEEL, LOOK, TILT and CHECK.¹ Feel is defined by the banknote substrate’s quality. Polymer banknotes featuring raised print effect on the picture of the first SPB Yang di-Pertuan Agong and words made of special plastic. Look involves examining the banknote under the light of a white bulb. A three-dimensional watermark portrait will appear, as well as a perfect see-through registration and a clear window. The security thread will appear in a continuous dark-colored line. Tilt involves tilting the banknote while holding it straight. Examine means examining the security thread and the coloured glossy patch for image and colour changes. Simple equipment may be used to check the banknote, except for certain security features, the Ultraviolet light device will not cause the paper substrate to glow. Micro-letterings will be easily apparent with a magnifying lens. By using the “FEEL, LOOK, TILT, and CHECK” principle, all current Malaysian banknotes counterfeits can be identified clearly without much trouble.

A person who is visually impaired has a vision problem that may not be corrected by wearing glasses. The difference between a blind person and visually impaired person is that the impaired is dim-sighted or visually challenged, not entirely blind, whereas the blind person is entirely blind.² The challenges experienced by the visually impaired people at conducting daily-life activities, particularly in operating a business, shopping and tasks involving banknotes handling, are similar to those experienced by blind people. A visually impaired person’s banknote transaction is usually handled by their accompanying trusted business vendor or a partner. However, this scenario puts the visually impaired person in danger of being duped in restricting the commercial activities by the accompanying partner or trusted business vendor.

The Bulgarian Cash Vision team developed the ‘b-note system’,³ a banknote scanner that helps visually handicapped Bulgarians recognize Bulgarian money. They developed a tiny box scanner that employs the camera sensor of a Raspberry Pi controller to record the bill’s middle section of an image using feature extraction algorithm to detect the minimal value (specific stamped marks at one of the banknote’s corners) and the value of the banknote currency. This banknote reader is not suitable to detect Malaysian banknotes because there are no engraved indications on Malaysian banknotes.

NantMobile Money Reader, developed by IPPLEX,⁴ allows users to aim their iOS device’s camera at a banknote and receive real-time denomination information. It accepts 21 different countries’ currencies, covering the US dollar, Singapore dollar, Australian dollar, etc. The Malaysian ringgit is also disclosed in the reader’s directory. However, this product is just an application software that allowed users to download and install physically on devices such as an iPhone, iPad, or smart tablet to use. The use of a touchscreen is inconvenient for blind and visually impaired people.

To assess Malaysian banknotes denomination, UTM researchers presented a banknote recognizer with sensor-based modality.⁵ The system employs an Arduino UNO as the processing component, which has a hefty physical architecture that makes it impractical for holding by consumers. Aside from that, the rule-based technique to identify the worth banknotes is intuitively established, with no classifier intervention or machine learning in the banknote interpretation. In 2018, the same researchers used Arduino Lilypad to improve the recognizer of banknote into a wearable device for identify the Malaysian Ringgit banknote.⁶ The TCS 34725 colour sensor data was fed into a suggested embedded decision tree classifier, which was then tested using 10-folder cross validation and compared to the k-Nearest Neighbour (k-NN) and Nave Bayesian classifiers.

The disadvantage of the Malaysian banknote readers proposed above are huge size, and no counterfeit detection. The huge size device makes it difficult to carry by visually impaired person. Therefore, the proposed Malaysian Banknote reader in this paper will relook into the embedded system design to solve the problem of the bulky size reader. Other than that, counterfeit detection will be embedded into the proposed Malaysian Banknote reader to detect the counterfeit of the banknote, safeguarding the users’ losses.

In this paper, a vision based Malaysian banknote reader has been designed to handle Malaysian banknotes for visually impaired people in order to improve the present Malaysian banknote reader and to meet the needs of visually impaired people when doing their regular business operations.

Different values of Malaysian banknotes are having different types of watermarks, for examples RM1 and RM5 required backlight mechanism, Tilting/rotating mechanisms were necessary for the RM10 and RM20, while ultraviolet light shooting mechanisms were necessary for the RM50 and RM100. The current developed banknote reader work is focused on recognized RM1 and RM5, with backlight mechanism and corresponding image recognition techniques.

The proposed Malaysian banknotes reader’s hardware components include a microprocessor for camera control, a speaker module and illumination. The primary operating idea is that the image of the banknote is captured by a camera, transmitted to the microcontroller for image processing. The developed image processing algorithm will trace out the region of interests: 1)see-thru windows, 2) Crescent and Star, 3) Perfect see though register, from the captured images and detect the watermarks features accordingly to decide the values and counterfeit for the inserted banknote. The detection results are then played as voice message on a mini speaker embedded on the banknote reader. This banknote detection system has a success rate of up to 89% in identifying the proper banknote value and counterfeit.

The paper is well ordered in following manner. The Malaysian banknote reader system model with backlight mechanism will be briefly detailed in Section II. Section III show the proposed image processing-based RM1 and RM5 Malaysian banknotes detection algorithm. Section IV comments same experimental result and lastly in section V, conclusion is future work are presented.

2. RM1 and RM5 banknotes reader system model

The system model for the RM1 and RM5 banknotes reading system is show in Figure 1. The banknotes detector is consisting of various parts and a slot of banknote insertion. The working principle start with the backlight platform with white light is turned on/off to captured two images of the inserted banknote, one with backlight and one with no backlight images. The two captured images are sent to microcontroller for image processing and check if the inserted banknote is a real RM1, real RM 5 or fake banknote/none of them. The results will be displayed on a speaker to allow the visually impaired person knows the holding paper notes.

Figure 1. RM1 and RM5 banknotes detection system model.

Figure 2. Mini speaker.

3. RM1 and RM5 banknotes detection image processing algorithm

The image processing algorithm for RM1 and RM5 detection can be divided into 6 steps:

Figure 3. Sample set of acquired RM1 banknote (a) image “Ba”, (b) image “Bb”, (c) image “Bc”.

Figure 4. Original and Gaussian Blur converted image.

Figure 5. HSV colour model.

Figure 6. Original image (left) and HSV image (right).

Figure 7. Track bars detect features in images.

Figure 8. Original image for RM1 (left side) and mask image (right side).

Figure 9. Original image for RM5 (left side) and it’s mask image (right side).

Figure 10. Four possibilities of RM1 correct detected 3 Region of interest.

Figure 11. Four possibilities of RM5 correct detected 3 Region of interest.

Figure 12. Bounding box of Region 1 (a) sample of RM1 (b) sample of RM5.

Figure 13. Sample of RM1 and RM5’s Crescent and Star images captured with backlight Off (left side) and with backlight On (right side).

Figure 14. Sample of successful Region 3 detection (a) WBa and WBb for RM1 (b) WBa and WBb for RM5.

4. Experiment result

The prototype of RM1 and RM5 banknote reader is constructed, as shown in Figure 15. The dimension for the banknote reader prototype is 235 mm (Length) × 142 mm (Width) × 135 mm (Height).

Figure 15. Banknote reader

Songket area in the real RM1 banknote is measured with dimension of 25 mm × 35 mm = 875 mm². The whole piece of RM1 banknote is with dimension 120 mm × 65 mm = 7,800 mm². Therefore, P_R1 for RM1 is 11.22% or 0.1122. Hornbill area in the real RM5 banknote is measured with dimension of 25 mm × 40 mm = 1,000 mm². The whole piece of RM5 banknote is with dimension 135 mm × 65 mm = 8,775 mm². Therefore, P_R1 for RM5 is 11.40% or 0.1140. Since the banknote reader is shared among RM1 and RM5 detection, hence the minimum P_R1 among the two is selected, and rounded to 0.11.

TR is Total Pixels in the Resized Image converted in Step 2, TR = 250 × 450 = 112,500 pixels. Hence, in step 5 Noise Object Exclusion part, any object with bounding box region smaller than 0.11 × 112,500 = 12,375 pixels will not be considered as Region 1.

The measured height of Songket’s pattern in RM1 banknote is 35 mm and the width of Songket’s pattern in RM1 banknote is 20 mm. Therefore, the height to width ratio of Songket’s pattern in RM1 banknote is 1.75. The measured height of Hornbill’s pattern in RM5 banknote is 43 mm and the width of Hornbill’s pattern in RM1 banknote is 23 mm. Therefore, the height to width ratio of Hornbill’s pattern in RM5 banknote is 1.87. To better classify RM1 and RM5 from one another, for RM1, Th_R1,RM1(min) is set to 1.69 and Th_R1,RM1(max) is set to 1.81; whereas for RM5, Th_R1,RM5(min) is set to 1.82 and Th_R1,RM5(max) is set to 1.93. Such setting is with the best tolerance gap to classify the two types of banknotes effectively. To get Th_R2,RM1, 100 different real banknotes of RM1s’ images were captured for 100 pairs of image “Bb” (backlight On) and image “Ba” (backlight Off). The Blue colour intensity value on the Crescent and Star’s sampled pixels were recorded and the difference between image “Bb” and image “Ba” were calculated and tabulated in the plots of no. of attempts vs. |Blue colour intensity difference between image “Bb” and image “Ba” |as shown in Figure 16. From Figure 16, it is shown that most occurrence happened in between Blue colour intensity value of 112 to 131. Hence Th_R2,RM1(min) is set to 112 and Th_R2,RM1(max) is set to 131.

Figure 16. Plots on no. of attempts vs.|blue colour intensity.

Difference between Image “Bb” and Image “Ba”|) for RM1

To get Th_R2,RM5, 100 different real banknotes of RM5s’ images were captured for 100 pairs of image “Bb” (backlight On) and image “Ba” (backlight Off). The Green colour intensity value on the Crescent and Star’s sampled pixels were recorded and the difference between image “Bb” and image “Ba” were calculated and tabulated in the plots of no. of attempts vs. |Green colour intensity difference between image “Bb” and image “Ba”| as shown in Figure 17. From Figure 17, it is shown that most occurrence happened in between Green colour intensity value of 114 to 135. Hence Th_R2,RM5(min) is set to 114 and Th_R2,RM5(max) is set to 135.

Figure 17. Plots on no. of attempts vs.|green colour intensity difference between image “Bb” and image “Ba”|) for RM5.

In step 3 of the image processing algorithm, if the clear window of a songket (for 1 Ringgit) or a hornbill (for 5 Ringgit) can be detected, the banknote is genuine; otherwise, it is counterfeit. In mask, the HSV values of the colour that are filtered out. Figures 18–21 illustrate the test run for some real and fake Malaysian banknotes. Experimental test was carried out with 100 pieces of real RM1, 100 pieces of real RM5 banknotes, 100 pieces of fake RM1 and 100 pieces of fake RM5 banknotes respectively revealed that the proposed banknote reader achieved around 99% accuracy for RM1 detection and around 78% accuracy for RM5 detection. The success rate of this system is up to 89% in recognizing the correct banknote value. From experimental test the threshold value of the acceptable colour intensity changes between the banknote image captured with and without backlight for RM1 (TH_B) from 41 to 57 and for RM5 (TH_G) from 60 to 78.

Figure 18. Real banknote RM1.

Figure 19. Real banknote RM5.

Figure 20. Fake banknote RM1.

Figure 21. Fake banknote RM5.

The total time for the banknote reader to complete 100 pieces of real RM1 banknotes detection is 1,148 seconds. Therefore, on average, the time required for one cycle of the banknote reader to capture in related banknote images, send to microcontroller to perform image processing and output the results on a speaker is 11.48 seconds.

Among the tested banknotes, for RM1, all the 100 pieces of the real banknotes and the 98 pieces of fake banknotes detected correctly. For RM5, 56 pieces of the real banknotes and all the 100 pieces of the fake banknotes detected correctly. To probe deep in to the failed banknote detection cases, confusion matrix is adopted.¹³ The four possible outcomes for the banknote’s detection scenario are diagnosed as list in Table 1 and Table 2 for RM1 and RM5 respectively.

Noteworthy attentions are placed on False Positive and False Negative cases, because these two cases may cause the visually impaired person losing credits in their business. For RM 1 detection, 2 banknotes detection cases, related to False Positive class and none cases related to False Negative class. Further analysed on these 2 False Positive cases, it is found that the fake RM1 banknotes were not placed properly into the Malaysian banknote reader (center of the banknote slot) and the Malaysian banknotes reader had mistreated some other areas on the corresponding fake banknote as the three Region of interest area (as shown in Figure 22), and this further identified the fake RM1 as the real RM1. To overcome this problem, normalized sizes were assigned on RM1 and RM5 at the Step 2 Algorithm (resizing image portion) to better locking the three Region of interest area.

Figure 22. Sample of RM1 False Positive Case.

For RM5 detection, 44 cases related to False Negative class and none of the case relate to False Positive class. Further probed on these 44 False Negative Class cases, it is found out that majority of the captured “Bb” images were not fully covered, as shown in Figure 23. The slotted RM5 banknotes cannot fully picture by the imaging tool, causing some of the regions of interest on the inserted banknotes (especially Region 2 and Region 3) cannot be detected. This is due to the size of the RM5 is much bigger compare to RM1. To overcome this problem, imaging area for the inserted banknote should be increased to cover the full banknote’s image. However, with existing imaging tool, this might need to be tolerance with a longer focal length with bigger size of banknote reader. Another alternative is to search for a wide view imaging tool to replace the current imaging tool for optimizing the current Malaysian Banknotes Reader’s size.

Figure 23. Sample of RM5 False Negative Case.

5. Conclusions

A Malaysian banknote reader employing image processing techniques was developed for visually impaired person to read and identify counterfeit on one Ringgit and five Ringgit Malaysian banknotes. The proposed portable banknote reader employed a visual type sensor to capture the inserted banknote image, sent to a Raspberry Pi controller for extracting the banknote’s watermarks and identify the banknote’s value. The detection result will be broadcasted on a mini speaker mounted on the banknote reader to help the visually impaired comprehend if it is a real one Ringgit, real five Ringgit, or none of them. The experimental results had proven that the proposed banknote reader is capable of completing several rounds of successful tries. In future, tilting/rotating mechanism and Ultraviolet light shooting mechanism can be embedded on the banknote reader to allow the visually impaired persons to cover the full series of Malaysian banknotes reading capabilities. The Malaysian banknote reader can also be expanded to support additional foreign currencies reading in the future. Aside from that, the size of the banknote reader can be improved, as well as the classifier intervention in the banknote interpretation. These issues will be resolved in the future.

Data availability

All data underlying the results are available as part of the article and no additional source data are required.