Monophonic Guitar Synthesizer via Mobile App

In the guild of guitarists, it is common to work with guitar synthesizers because the emulation of a great variety of sounds that are produced by different musical instruments, starting from just playing the guitar, which means, a piece of music is played with a guitar, but other musical instruments are actually heard such as, a saxophone, a violin, a piano or percussions, depending on the instrument that has been selected. The problem that arises in this article is that synthesizers are expensive and due to their size, the transportation of the equipment is often impractical. As mentioned, the development of a mobile application that has the function of a monophonic synthesizer is proposed as a solution. In this way, the cost is greatly reduced, and additionally, the user is able to install the application on a mobile device with Android operating system and connect it to an electric or electro-acoustic guitar through an audio interface; obtaining as a result, a functional technological instrument by offering guitarists an alternative with respect to conventional synthesizers. The construction of this application used the Fast Fourier Transform Radix-2 as a signal recognition algorithm, which allowed obtaining the fundamental frequencies generated by the guitar, which were transformed into MIDI notation and later used in sound emulation. Keywords—Monophonic synthesizer; guitar; sound emulation; mobile application


I. INTRODUCTION
The advent of tablets and smart cell phones has opened a range of possibilities in all areas of knowledge. In the field of music, technological development has occupied a privileged place by having mobile applications that help in tuning musical instruments, in measuring time to practice, in recording and editing music, among other aspects.
The rise of technology has allowed the creation of tools in order to support musicians in a vast diversity of ways. The tool that is interesting to highlight in this article is related to the emulation of monophonic sounds of musical instruments or other types of sounds, which is obtained through the use of synthesizers. The Royal Academy of the Spanish Language defines synthesizer as: "Electronic musical instrument capable of producing sounds of any frequency and intensity and combining them with harmonics, thus providing sounds of any known instrument, or sound effects that do not correspond to any conventional instrument" [1]. Synthesizers are very useful devices because when connected to the guitar, are able to provide a great variety of sounds, increasing the possibilities of musical interpretation of the guitarist.
Two of the problems that arise are its portability and its high cost; Due to the mentioned, based on computer science and mathematics, it was decided to develop a monophonic synthesizer, using a mobile application, which allows the guitarist to use a synthesizer through a mobile device such as a smart cell phone or a tablet, which can be transported easily, without having to make an additional expense.
Music can be classified into two main categories: monophonic and polyphonic. Monophonic music is made up of a single melodic line, which means, only one musical note sounds at a certain time, while polyphonic music is made up of more than one melodic line, which means, two or more musical notes sound at the same time [2]. The present study focuses on monophonic music.
The guitar is a musical instrument that allows its player to express musical notes in different ways. The guitar, like other instruments such as the piano or the violin, generates analog sounds, which need to be converted to digital sounds in order to be read by the computer. From this digitization, it is possible to apply signal recognition techniques for different purposes and in this case, to detect the fundamental frequencies generated by the guitar. A transformation was applied to these frequencies using the notation mentioned in the Musical Instrument Digital Interface (MIDI), to obtain a discrete representation of the musical notes, which are defined in an interval that goes from 0 to 127, where each number corresponds to a musical note.
The overall objective of this study was to develop a portable tool that allows the guitarist to emulate monophonic sounds of other musical instruments or other types of sounds using the guitar, all this through computational and mathematical techniques. For this purpose, the following specific objectives were proposed: • Recognize frequencies generated by guitars making use of hardware and software resources of mobile devices.
• Transform frequencies generated by guitars to MIDI notation.
• Emulate monophonic sounds of musical instruments or other sounds based on the obtained MIDI notes.
This article is divided into 4 sections. The second section shows some synthesizers available on the market, as well as some application programming interfaces that have been developed in order to recognize signals and to support the execution of MIDI sounds. The third section deals with the theoretical references about the recognition of monophonic sounds and the transformation of frequency to MIDI notation. The fourth section shows the methodology used for the 409 | P a g e www.ijacsa.thesai.org development of the monophonic synthesizer following the stages of the incremental Software Engineering model. Subsequently, the results obtained from the tests carried out are shown and finalized with the conclusions.

II. RELATED WORK
At present, there are independent synthesizers for guitar, which means, mounted on electronic circuits, some of them are: MEL9 [3], SY-300 [4], GR-55 [5], among others; However, when it comes to mobile applications that perform the function of a guitar synthesizer on the Android operating system, there are no formal alternatives to it.
On the other hand, there are some application programming interfaces (APIs) aimed at the Android operating system, which can be useful for the construction of a synthesizer, for example, those that serve to perform frequency recognition and those that serve to execute MIDI sounds, some of these interfaces are: TarsosDSP [6] and MIDI Driver [7] respectively.
In the present mobile application, it was chosen the frequency recognition through the Fast Fourier Transform Radix-2 algorithm, according to [8], making use of the native Java development kit, while for the execution of MIDI sounds, the model that was done in [7] was retaken.
III. THEORETICAL ASPECTS In this section, reference is made to the techniques used to recognize monophonic sounds and, on the other hand, the digital interface of musical instruments (MIDI) is presented, which shows a notation that serves to discretize frequencies.

A. Monophonic Sound Recognition
Due to in this article the monophonic sounds generated by the guitar are taken as a basis, some algorithms that can be used for the recognition of monophonic sounds are specifically mentioned.
There are mainly two approaches that are used to perform monophonic sound recognition. One of them consists in analyzing the signal samples in the time domain, and the other in analyzing them in the frequency domain. A widely used method in the time domain is autocorrelation, which compares a signal with delayed versions of itself at successive intervals to find the highest amplitudes within the signal and measure the distances between them. Through these distances the period of the wave can be inferred, and with it, the present frequency of the monophonic sound can be detected. On the other hand, there is the analysis of signals in the frequency domain, where algorithmic implementations of the Discrete Fourier Transform are used, with which a set of frequency intervals and their amplitudes are obtained. A simple way to detect the frequency of the monophonic sound present quickly is to select the frequency with the greatest amplitude [9].
In this article, the signals in the frequency domain are analyzed, so some algorithms that can be used to recognize the monophonic sounds generated by the guitar in that domain are specifically mentioned.

B. Fast Fourier Transform (FFT)
Fast Fourier Transform is an efficient mathematical implementation of the Discrete Fourier Transform (DFT), which is a particular case of the Fourier Transform for sequences of finite length in which the spectrum is evaluated only in a few specific frequencies, and therefore, a discrete spectrum is obtained [10,11].
Over time, several FFT algorithms have been developed such as: prime factor, split radix, vector radix, split vector radix, Winograd Fourier transform, etc. [12].
The FFT algorithm used to develop the mobile application of the monophonic synthesizer is the Radix-2. The Radix-2 algorithm is considered the most used for the FFT calculation, it works when the number of data samples is a power of 2, in case the number of samples does not satisfy this criterion, the missing spaces are filled with value 0, this does not alter the calculated frequency spectrum. The input and output of an FFT are expressed in complex numbers, in this case, for its implementation, two arrangements are accepted to store the real and imaginary components in them, when using the recording tools of mobile devices, all the bytes of audio information are obtained, which are used within the real components, while the imaginary components are always filled with zeros, the output of the algorithm is contained in two other arrays, one corresponding to each type of component, where the frequency spectra are stored, since only the actual samples are used for input, only the first half of the components need to be analyzed. Each component of the frequency spectrum is related to the previous component, since it is the sampling frequency divided by the number of samples of the FFT [8].
In order to recognize monophonic sounds, the power or amplitude spectrum (Xp) is analyzed, selecting the frequency f with the greatest amplitude, which is calculated through the sum of the squares of its real (Xreal) and imaginary (Ximag) components, as shown in (1), 2 (1) The FFT algorithm used to develop the mobile application is Radix-2, which means that it must work on a group of samples whose number is a power of two [8].

C. MIDI
Musical Instrument Digital Interface (MIDI) is a music notation system that allows computers to communicate with musical synthesizers. MIDI files contain instructions to create the pitch, volume and duration of the notes, this based on a sequence of events called: note_on and note_off [13].
Musical notes are not encoded by their names, instead numbers from 0 to 127 are assigned as shown in Table I. For example, the number 57 corresponds to a musical note A with a frequency of 220 Hertz (Hz).
Equation (2) shows the transformation of frequency in Hz to MIDI note, 410 | P a g e www.ijacsa.thesai.org Where round is a function of rounding to one digit, factor 12 is the resulting linear pitch space per octave, factor 69 is note A (440 Hz), which is taken as reference, log 2 is used according to the logarithmic pitch perception in humans and the variable f is the input frequency that will be converted to a MIDI note [14].
General MIDI is a standardized specification for electronic musical instruments that respond to MIDI messages. General MIDI was developed by the American MIDI Manufacturers Association (MMA) and the Japan MIDI Standards Committee (JMSC) and first published in 1991 [16].
Within the general MIDI specification 128 sounds of musical instruments or other types of sounds are included, and these are divided in sections such as: Piano, Chromatic Percussion, Organ, Guitar, Bass, Strings, Ensemble, Brass, Reed, Pipe, Synth Lead, Synth Pad, Synth Effects, Ethnic, Percussive and Sound Effects which are used in this mobile application and are shown in Table II. 411 | P a g e www.ijacsa.thesai.org

IV. METHODOLOGY
The mobile application was developed using the phases of the incremental Software Engineering model, which applies linear sequences in a staggered manner as the calendar of activities progresses. Each linear sequence produces deliverable software increments [17]. The diagram in Fig. 1 describes the stages used for this development.
According to what Mall [18] points out, first of all a simple system is built and delivered which implements only a few basic characteristics. During a few successive iterations, improved versions are deployed and delivered, until the desired system is finally realized.
The software requirements are first divided into several modules or features that can be built and delivered incrementally. This is graphically represented in Fig. 2.
Returning to what Pressman and Mall [17,18] point out, 3 modules were created as part of the development of the mobile application.
In the first module, the recognition of the frequencies generated by the guitar was carried out, for which the Fast Fourier Transform Radix-2 algorithm was applied, making use of the hardware and software resources of mobile devices. When playing a musical note with the guitar, the fundamental frequency corresponding to said musical note was obtained in real time, showing it on the mobile application interface.  In the second module, the transformation of frequencies to MIDI notes was carried out, for which equation 1 shown in the MIDI section was applied, obtaining as a result, discrete values of frequencies within a range from 0 to 127, showing the aforementioned transformation together with at the fundamental frequency in the mobile application interface.
The third module allowed to emulate the monophonic sounds of musical instruments or other types of sounds, based on the MIDI notes obtained. The note_on and note_off events of the digital interface of musical instruments were used to execute and stop the MIDI notes. Consequently, the moment the user plays a musical note with the guitar, the note_on event is activated, with which the sound chosen by the user within the 128 included in the mobile application must be heard; for example, a saxophone, a trumpet, a violin, among others; whereas when there is an absence of sound, the note_off event was activated to keep the mobile application silent.
For the construction of all these modules, the Java programming language was used together with the Android Development Kit (SDK). Fig. 3 shows the block diagram of the mobile application. As can be seen, an audio input is required, which goes through a sampling process to obtain the discretized signal, making use of the audio recording tools offered by the Android operating system. Subsequently, the filter allows to eliminate the peaks of the signal, which means, it eliminates the unwanted noise, and then to apply the Fast Fourier Transform and thus obtain the fundamental frequencies. From these fundamental frequencies, the transformation to MIDI notation is carried out. From this moment, the sound chosen by the user can be emulated through the execution of the note_on and note_off events. Fig. 4 represents the operation of the mobile application. In order to use it, it is necessary to connect it to an electric or electroacoustic guitar through an audio interface, the last one is connected to an audio output device such as an amplifier.

V. RESULTS
The mobile application has been developed using the Java programming language in order to run on mobile devices with the Android operating system. The tests were carried out on a device from the Motorola brand, a Moto G7 Plus model with 64 GB of storage and 4 GB of RAM, with the Android 10 (Android Q) operating system installed.
The work was carried out with a sample of three guitarists to carry out the tests of the operation of the mobile application. Two of the guitarists used an electric guitar and the remaining guitarist used an electro-acoustic guitar. The musical instruments were connected to the mobile application through an iRig 2 audio interface. The guitarists selected in the list of the main interface of the mobile application, several of the 128 sounds that were available to be emulated and they played the guitar obtaining the sound expected. Additionally, they were able to observe the frequency of the note played, as well as its transformation to a MIDI note in real time.
The mobile application was subjectively evaluated with the feedback obtained from the guitarists, who pointed out the great usefulness of this mobile application because it broadens their possibilities of interpretation with their musical instrument, without requiring more than their mobile device and an audio interface. On the other hand, they mentioned that they noticed a slight latency between the moment they played the guitar and the emulation of the chosen sound, which could be reduced using a low-level programming language. Two examples of the mobile application in operation are shown in Fig. 5 and Fig. 6.  414 | P a g e www.ijacsa.thesai.org

VI. DISCUSSION
The mobile application was developed with the Java programming language. Due to Java uses a virtual machine (Java Virtual Machine, JVM), which processes the instructions before being executed [19], a considerable latency was obtained in the results. To those who want to return to this article for their research, they are advised to use a lower-level programming language such as C or C++, with which the latency would be reduced considerably and obtain better results, because the instructions are executed directly. Guitar effects pedals commonly use recommended programming languages [20].

VII. CONCLUSION
The development of mobile applications has acquired great relevance due to the variety of areas where they can be used. As mentioned, there are applications that allow communication between people, planning travel routes, requesting food delivery and entertainment, to name a few examples. Additionally, the applications have allowed people to count with tools that provide the opportunity to explore, grow and develop in other areas such as music, which contributes to their comprehensive training. The mobile application presented in this work benefits this aspect because the user has a synthesizer in the palm of his hand that is capable of emulating monophonic sounds of different musical instruments or other types of sounds with a guitar. This synthesizer, unlike the ones available on the market, is affordable, easily transportable and usable anytime, anywhere. Applications as the one described above allow the practice of music to be accessible to huge number of people.
It has been considered as future work, to make this mobile application a polyphonic synthesizer, where to obtain the polyphony generated by the guitar, the frequency spectrum obtained by the FFT Radix-2 algorithm will be taken up, and search and decision mechanisms will be used. It is also planned to adapt the mobile application to a lower-level programming language like C++ for lower latency.