Fault Tolerance Smart Egg Incubation System with Computer Vision

—Reliability of incubators is one of their most important specifications. Development of wireless, cloud and computer vision based technologies gives new possibilities for work process control and increasing fault tolerance. Regardless of whether the hatching is in the field of mass production or in the breeding of rare species of birds, detecting a critical situation and sending timely notifications can prevent serious losses. Experience shows that network isolated solutions are not reliable enough and good management requires complex algorithms that are beyond the capabilities of a local, single controller. Even with the duplication of some sensors and actuators, incubators without external connection are high risk due to the fact that their controller is a central point in the architecture and can fail, leaving the farmer with no alert about the accident. The report presents a solution that uses periodic checks from cloud structures on the condition and operability of the incubator. In parallel, a video surveillance system analyzes the internal environment and the condition of hatching chicks. When potential and real risks occur, the system sends notifications to the responsible persons even to his or her wrist. Additionally, the proposed smart egg incubation methodology has been found to reduce the amount of time required for farmers to oversee the incubation process by up to 50%, allowing them to focus on other important tasks while still ensuring optimal hatching conditions for their eggs. Overall, the proposed methodology offers a significant improvement in egg incubation efficiency and reliability, with potential applications in both commercial and personal settings.


A. Defining the Research Problem
Smart farming, like industrial production, follows the steady trend of replacing manual human labor with automated systems. Innovative solutions manage not only purely mechanical activities but also digital ones implementing artificial intelligence and computer vision in many systems to generate feedback for control and corrections of processes during their operation. In this way, the possibility of human error is reduced to the lowest possible values and the dependence on the labor market is minimized. The ultimate goal that is achieved through these trends is a significant increase in efficiency and respectively productivity. First smart farming solutions that have been developed include systems with controllers, sensors and actuators, but without video cameras.
The prerequisites for the development of computer vision based smart farming include:  Offering cameras with high matrix resolutions and the ability to capture many frames per second on a lower price.
 Development of new network standards for highthroughput wireless communications.
 More accessible cloud technologies that provide the ability to process more complex algorithms in image analysis.
 Ability to customize processing load distribution between on-premises and cloud structures.
 The trends for a non-expansive increase in a farm production.
It is a hard task to list all the examples in this area, but some of them may be referred to as a representative sample. In crop production, a large group of computer vision based applications aims to recognize the fruits of plants in order to pick them [1,2]. Some others analyze pictures of plant leaves to find diseases and determine their condition [3,4,5]. The outcomes of such systems can be used for appropriate subsequent treatment (spraying) of the plants [6].
There are no less in a number of examples of computer vision systems in animal husbandry. Examples for animal farming include counting of silkworm eggs [7]. Sometimes in animal farms, the processes of image acquisition and processing can be significantly complicated due to the more intensive motor activity of animals compared to that of plants. Such case with pigs is described in [8]. It describes the algorithms for separating individual animals from a group in which they are close to each other. Of course there are again examples of recognizing health problems of animals from the processed images [9,10]. Feeding process control is also a possible target of the computer vision system in a smart farm [10]. This paper explains how to recognize pigs with low weight. Poultry farms are no exception to the implementation of intelligent machine vision solutions. In this scenario infertile egg detection is a key feature [11].
Some interesting applications corresponding with smart farming also exploit computer vision methods. Baoming Shan uses image analyses of eggs in vaccine production from embryos [12]. Fertility detection is again targeted in the results, although the field of use in this case is pharmacy. www.ijacsa.thesai.org A large number of papers [13,14,15,16] oriented to a human incubators describe machine-vision mechanisms for monitoring the state of babies. Different type of cameras can monitor movements, temperature and persistence of a newborn. From the recorded movements, it is possible to draw conclusions about certain behavior according to a health condition and thus detect critical situations. The described systems are traditionally used in combination with standard sensors and controllers that upgrade the information from the images.
All listed examples above lead to the conclusion that computer vision can be very helpful into the hatching process and may increase the reliability, by protecting the eggs and new chicks from injury and exposure to adverse conditions in the incubator's internal environment.

B. Importance of the Problem
Today the share of small farms that breed several species of animals is decreasing compared to that of large and strictly specialized in certain breeding. Therefore, methods and devices for hatching a large number of chickens are sought in poultry farming. In case of hatchery failure, this approach makes the potential losses significantly greater.
Apart from hatching for mass production, with the right settings, the small, modern, intelligent hatcheries can also be used for breeding exotic species of birds and reptiles. This scenario is also associated with a high risk of failure because eggs are usually very rare.

C. Purpose and Tasks of the Research
In order to reduce the possibility of harmful consequences in the situations described above, it is necessary to increase the reliability of the hatcheries using modern methods. This main purpose can be divided into the following tasks:  Selection of building modules.
 Implementing cameras in the hatcheries so that operators can monitor the interior.
 Creation and implementation of a computer vision system in the hatchery.
 Preserving the functionalities for reliable notifications of the operator in a problematic situation.
 Analysis of possibilities and selection of a model for building the connections between the devices, the cloud environment and the operators.
 Implementation of a prototype corresponding to the selected model.

D. Hypothesis
Cloud technologies can be a key tool to increase the reliability. The integration of a computer vision system can be realized locally through a controller and a smart camera, but such solutions are too expensive. On the other hand, using a microcomputer and camera is an alternative, but using an operating system can generate new potential points of failure (opportunities for viruses, attacks, sudden reboot problems, etc.). Cloud structures provide an optimal solution because they do not require computing potential from the controllers, but possess the necessary for the implementation of the functions desired by the user.

E. Expected Benefits
The results of research, described in this paper try to offer a model for building a modern incubator with maximal fault tolerance, level of automation, flexible control and visual feedback for the users. Suggested devices should also allow comfortable parameters observation (charts) and optimization of the processes. Following this line of thought can be defined a new system of incubators as Fault Tolerance Smart Egg Incubation System with Computer Vision (FTSEISCV). The reader should note that this is a development of another study on smart incubators. Prior publication on development of Smart Egg Incubator (SEI) can be seen at [17].
The main contributions of the proposed smart egg incubation system include advanced features such as microcontroller-based control and monitoring, camera-based hatching detection, and precise regulation of temperature, humidity, and airflow parameters, which collectively provide farmers with an efficient, reliable, and user-friendly solution for egg incubation. Additionally, the system's ability to reduce the amount of time required for farmers to oversee the incubation process by up to 50% can have a significant impact on productivity and profitability, while also improving hatching success rates and reducing the risk of losses due to incubation-related issues.

F. Organization of the Paper
The paper contains an overview of historical development of incubators in the second part and analysis of used models up to the moment in the third. Next fourth part represents the proposed, improved model, selected hardware components for the experimental prototype, used software and cloud structures. User interface and control management are also explained in this part. The paper finishes with conclusions and references.

II. OVERVIEW OF EXISTING SOLUTIONS ON THE TOPIC
The first incubators from 20th century were made from a wooden box with a pot-like container attached to the side ( Fig. 1) [18]. They included a thermometer and required almost constant monitoring by farmers.
Electronic analog-element managed systems increased the automation. They included humidity and temperature sensors www.ijacsa.thesai.org and actuators like heaters, fans, evaporators and mechanisms for eggs rotation.
After IC chips implementation, some advanced and more complicated functions were able to be performed. Specific to this period is the transition from analog to digital signal processing.
Until a few years ago the only accessible option to increase reliability was realized by duplicating individual elements of the incubator. Thus, when one of the elements fails, the backup one takes over its functions. However, these solutions often do not work if the controller module fails and therefore require regular checks from operator to prevent disaster.
First notification developments were realized through mobile network gateways. At this moment were implemented alerts for the operators, when incubation parameters exceed the preset limits. Unfortunately, if the cellular network is not available or if the user is temporarily unavailable (due to personal reasons sometimes), the notifications are not repeated and thus the desired level of trust is not guaranteed.
Programmable controllers allow full automatic control during the whole incubation. In addition to temperature and humidity, sensors in modern incubators also measure the percentage content of oxygen, carbon dioxide and ammonia compounds in the air.
The limitations of existing egg incubation systems, such as poor temperature and humidity control, inconsistent egg rotation, and limited hatching success rates, motivated the development of the proposed smart egg incubation system with advanced features like microcontroller-based control and monitoring, camera-based hatching detection, and precise regulation of temperature, humidity, and airflow parameters. The next step in the upgrade of the modern incubators is realized with the expansion of the World Wide Web and their digital transformation in the industry [19] into Internet of Things devices. Initial attempts in this direction were aimed to monitor the processes and environment and control them over the network, only by the operators. At this point, in addition to cellular networks, notifications could now be sent and received in parallel or alternatively via the Internet. This kind of device operation can be defined as a third layer of the mentioned model above.

III. OUTCOME OF THE RESEARCH ON THE EXISTING SOLUTIONS AND A PARTICULAR CONCEPT
As an outcome of research on the existing solutions, can be made a conclusion that the first incubators were built only on one layer, namely a device level. Then the users did not have own terminal devices for monitoring and control, and also cloud structures did not exist in this period. On objective reasons, it was necessary for the operators to be constantly near to the hatcheries. Later solutions were constructed over twolayer model. It contains client and device layers, because users already own a primitive mobile device like pagers, 1G or 2G phones. Connections between levels in this case were straight and were built primarily over mobile networks.
Thinking about the possibility of incubators being connected to the Internet and controlled by client devices, such as computers and smart phones, cloud technologies were chosen as the most intelligent solution to achieve the goal set in the research. The Cloud comes in as an intermediate layer (between Device and Control) to store and process data, as well as provides services such as video streaming, notifications, routines deployment, and more. Therefore, the chosen model for creation the FTSEISCV, is an adaptation of the three-layer model known from the IoT architecture, namely Application-Network-Perception model [20,21].
In the present research an adaption to Device-Cloud-Control was realized. Device stands for the new smart incubator, Cloudfor the cloud solutions and Controlfor the farmer's computers and mobile devices.
The adapted three-layer model shown in Fig. 2 incorporates the additional cloud layer that enables the realization of the IoT concept and a wide variety of additional services as a part of next step in global network development [22]. The cloud layer mediates between user devices and the managing controller part. The connections between the layers are now realized with priority over the Internet. The three-layer model is of increased complexity, but provides functional flexibility in many ways, including improved reliability. Therefore, the three-layer model is the choice on which is based this study. Following modern trends, there is no need to try implementation of new layers, but improvement of the current ones and optionally dividing any of them into different parts. On device level except sensors, controllers and actuators, additional imaging devices can be introduced. On a cloud level a solution can be to migrate some of the functionalities in order to prevent overload in a controller part (device level) and to provide an external point of operability testing. Cloud layer also takes care to guarantee secure notification to the user.
Here are a few examples of state-of-the-art egg incubators that you can find online: By exploring these and other state-of-the-art egg incubators available on the market, the reader can gain a better understanding of the features and capabilities that are available in today's incubation systems, and compare them with the features and capabilities of the presented here smart egg incubation system.

IV. DEVELOPMENT OF FTSEISCV
The development of the whole system includes creation of its conceptual model, device architecture, cloud functionalities and control applications. All devices, cloud components and applications are designed to work as one unified system. The basic part of every system is its model. So, first step is to emphasize the model of the new system.

A. Model
Creation of conceptual model is based on the scientific analysis, conducted and described in the previous point (Fig. 3).
It presents a number of SEI in the Device layer that connects to the cloud solutions via the Internet. On the other side are the control devices in the Control layer that communicate with the cloud solutions. As a result, the farmer has all the data and functions necessary to control the SEI.
Prototype planning details include a segmentation of the components along the layers of the adopted model and an organization of the connections between them in an optimal way. After extensive analysis and partial approvals, the model presented in details on Fig. 4 has been created.

B. SEI Architecture and Components
Selection of the hardware for the FTSEISCV in the device layer is based on components that are available relatively cheap and at the same time does not compromise on their reliability and performance. The popular development environment and the support of many already created libraries for the components are other main criteria.
In practice, one of the innovations of the developed system is the provision of fault tolerance, which does not rely only on failsafe elements. On the second level, it is guaranteed by the cloud structures and the well-chosen connections between the components. Additional cloud applications ensure this functionality.  There are three DHT22 temperature and humidity sensors. They fully cover the requirements for accuracy, working range, consumption and supplying digital values to the controller. Two are located inside to monitor the homogeneity of the hatching environment. The third one is placed outside to increase the attention of the operator in the case of too great differences in the external environment compared to the internal one, because in such situations they imply increased risks and overloads. www.ijacsa.thesai.org Capacitive moisture sensor located outside works as a level meter gauge in the outer water container of humidifier, which is used to increase the humidity if necessary. Capacitive humidity sensors are more suitable than resistive ones, because when measuring the resistance between the individual electrodes electrolysis occurs and they corrode and fail.
Contactless infrared temperature sensor is placed inside and is directed at the shell of one of the eggs. It acts as system feedback, supplementing the readings of the DHT22 sensors and increases reliability.
The CO2 gas sensor is most actively used in the hatching phase and aims to notify the farmer of a potential suffocation danger to the chicks. When they have already hatched, they become active, breathe rapidly, releasing carbon dioxide accordingly.
Another sensor is positioned on the egg rollers and provides feedback for rotation process. It is an encoder that switches its logic states between of zero and one with a certain frequency determined by the speed of rotation. To work reliably, it must be well sealed and waterproof.
The SD Card Module shown on the schematic is optional and can be used to store the data read from the sensors before being sent to the cloud database. When this module is not used, these functions can be performed by pre-allocated partitions of controller memory.
As mentioned above, the controller occupies a central place in the hardware architecture. That is why its choice is one of the most important to achieve the planned global goals. By using Texas Instruments or Nordic Semiconductors devices for example, can be achieved really good final solutions, but the value of the incubator in these cases will increase significantly. On the other hand, NodeMCU (ESP8266) is also a possible solution, but the potential for future development and its lower reliability when performing more functions; it is not the best choice. Compared to it, ESP32 offers not one but two computing cores, double the clock speed, more input-output pins, more SRAM memory and an additional Bluetooth channel for communication in parallel with WiFi. ESP32 is also suitable because of its ability to use the ArduinoIDE programming environment, which is free and has a wide variety of already developed and available libraries.
To implement the computer vision system, it is necessary to select a suitable camera. Variants with a USB connection are not suitable, because the ESP32 controller will have to transmit simultaneously the video stream and data from the sensors via WiFi. This can create conflicts and competition for the communication channel and thus reduce reliability. The purchase of smart cameras with their own computer vision system and/or Wi-Fi connection to the cloud structures is an inappropriate optional solution because it can again lead to an excessive increase in price and for one reason or another, not covers the requirements for full compatibility between all system components. In accordance with the stated requirements, the best choice remains ESP32 Cam. Through its Wi-Fi channel that is parallel and independent compared to that of the ESP32 controller, the camera can stream video to the network. If direct communication with the controller is required for data transmission, one of the additional GPIOs or the Bluetooth connection can be used. An additional functionality that may not be used is the SD card slot with which the camera is equipped. The ESP32 Cam is available on the market with variable focal length lenses that are selected according to the specific internal hatchery design.
AC/DC power supply 2A 5V is used for the controller, camera and sensors. Its rated power significantly exceeds the total power consumed. An additional V8 mobile power expansion board with its own battery is selected to protect the controller and camera from unwanted reboots. It operates as UPS in the event of a central power failure. Thanks to it, in such a critical situation, the system can continue to monitor the readings from the sensors and send data if there is an available access point, even though the actuator parts are not active.
The actuators on the Fig. 5 are light source, outside airflow fan, egg rollers, humidifier with fan and heaters with fan. They all use 220V mains power supply. Heaters maintain the necessary internal temperature, and their fan aims to spread the heat evenly inside. In conditions of excessive heating or too high humidity, the internal environment in the hatchery is normalized by means of the external airflow fan.
In contrast to heaters, practical tests have shown that humidifier should be placed outside the incubator. Its fan conducts the moisture evaporated from the water through specially designed pipes to the interior.
Egg rollers are placed on a common frame and are rotated simultaneously by a stepper motor attached to a worm gear mechanism. The position of its shafts can be adjusted according to the size of the eggs being incubated. Every computer vision system needs appropriate lighting source. Therefore, the camera in SEI needs lighting with a stable intensity to work correctly. In the experimental prototype, this role is performed by a constantly glowing 7W LED lamp.
The connection between the controller and the actuators is performed by 4 electronic solid state relays. They are significantly more reliable than electro-mechanical ones because they do not have the possibility of the contacts selfwelding during operation.

C. Functional Capabilities
Blynk cloud solution has been chosen for the practical implementation of the system. Blynk allows registration of multiple devices. In this particular case, the SEI software that works with the main ESP32 controller and the auxiliary ESP32 Cam was developed. Blynk supports the device with proper version of the software automatically. The software communicates with Blynk to transmit the values from the sensors, the states in which the incubators are at a particular moment, the video stream and others.
The system is designed to be controlled by the farmer's control devices (computer and mobile phone). For this purpose graphical user interface in Blynk application has been developed for mobile devices and computers (Fig. 6). It allows the user to start, stop and pause the incubators, to set the values of temperature, humidity, critical temperatures, critical www.ijacsa.thesai.org humidity, etc. that the incubators have to control, to watch the video stream from inside the incubator and many others. The GUI allows the user to send commands to the SEIs trough a terminal window in order to control them or to receive specific information for their states (Fig. 7).  Telegram has been chosen as a notification platform. When critical notifications appear, they are sent from the SEIs through Telegram to the mobile phone and eventually to the smart watch of the farmer. At that moment the user can react immediately to prevent damage of the production.
A special application has been created to check the operability of the devices at any time. If it detects a problem with a device it sends a proper notification through Telegram to the farmer.
An algorithm for vision inspection of the eggs has been developed and implemented to work in the ESP Cam modules. Its purpose is to detect the moment of hatching a chicken from an egg. If the module detects such an event, it sends a proper notification to the farmer. Then the farmer can observe the incubator on the video stream or at place (Fig. 8).

V. CONCLUSIONS AND FUTURE WORK
A real prototype has been assembled for approbation of the proposed FTSEISCV model. Partial and complete tests show stable results, proving a high degree of reliability, functional conveniences and a high degree of automation that guarantees a minimal commitment of users to the incubation process. By multiplying through a clustered approach, the system can be used on an industrial scale without compromising the intended performance.
Further researches will be focused on finding an optimal position of multiple cameras for the target area, management of the parallel video streams, improvement of computer vision algorithm and implementing of proactive features in the incubation control.