Andranik Meliqsetyan responds to the article “Smart Continuous Improvement – A ML Case” that appeared in Issue 1/2024 of this Journal.

Public debates in a written form are one of the aims of this Journal. A written debate has enormous advantages over oral debates. Firstly, it is public - meaning that everyone can read and benefit. Secondly, you need to augment your knowledge with additional reading to give your arguments a certain depth, and as such enriching your own knowledge. And thirdly, when you debate in a written form you must formulate your thoughts clearly and lay them out in a certain logical sequence.

Also, it must be said that the articles in the Journal are personal opinions, as such they don’t necessarily reflect the truth on the subject. Truth could be reached only through critique (debate) and practice. Hence, any critique and debate of the articles will only further enrich both us and the Journal.

This article is a must read by all who are involved in SW. This will give us a sense of the direction that we must take regarding SW skills and trainings.

This is a response to the article “The Smart Continuous Improvement - A machine Learning Case” that appeared in the 1^st edition of the EC Journal (Issue 1/2024).

Some Criticism

When, I first read this article about the smart continues improvement – a Machine Learning Case and the concept brought there, I realized we have enormously big gap in leveraging the technology that meant to solve that type of challenges in the type of a systems we design develop and manufacture here at EC.

We never focused on SW enough because HW and the associated SW which is mainly on the System level and deals with drivers and the operation of the devices to conduct measurements, but as we are evolving in building more systems and focusing on the reliability of the systems meaning that you have already developed a system that you maintaining and releasing a new version over the years and you want or not you must focus on the SW improvements and if you want to be competitive you need to have that features for connectivity, data aggregation and ML in place.

SW industry is there already, they are already designing and developing a solution that has built in mechanism for reliability and self-correction, and believe or not you are using their solutions daily, yes, those web applications, your phone app, that are running 24/7, some of those are having new versions of SW released every few days and you even not noticing that they are updated the SW. And behind the scenes there is a very well-established and orchestrated CI/CD process of developing SW, testing, and deploying it without interrupting the daily job of the users. CI/CD is the solution to your problem and in Armenia IT industry has mastered it very well, the same IT industry to which we have high level of ignorance and sense of competition but in my opinion, time is coming that we need to get closer to IT to master world class SW developed moving forward.

Well, you might be asking how to start embracing the world class SW developed for our cases which might need a more conservative approach as the level of needs are different. And NO you cannot put ML into your systems to miraculously solve the SW bug and prevent SW from crashing right away. That’s a state-of-the-art solution that requires a list of prerequisites built into your systems and needs to have full-time developers with the right skill sets.

CI/CD in a nutshell

When designing an SW for your systems you need to keep in mind the principles of CI/CD (Continuous Integration/Continues development). It’s not easy to get there, you should start step by step if you want to get in world class software development, leveraging connected systems data and driving continues improvements and efficiency through ML in your applications. CI/CD is a broad area, and the specific tools and practices can vary based on the technology stack, team preferences, and organizational requirements. However, the goal remains the same, automate as much of the software development process as possible, increase efficiency, and reduce the chances of human error. Here you can see below the main aspect of CI/CD and tools and approaches used there. Keep in mind you most probably do not need them all in this stagr but some of the approaches are must to embrace. Full CI/CD is typically well established in SW R&D teams.

CI/CD high level breakdown

• I am not going to breakdown all the aspects of the CI/CD there are lot of good trainings on this in YouTube and anywhere else, just brought few things from CI/CD that I think is important to embrace as you redefine your approach to legacy code development.

1. Source Code Control management a. Source Code Control is absolute necessity for any SW developer, you must use Source Code Control even if you are only SW developer in the team, does not matter what language you use for coding you must be doing source code control to track changes in the codebase, maintain code versions, and manage contribution from multiple developers. Various tools are being used for this, majority of devs are using Git with the GitHub, GitLab, Bitbucket etc.

2. Continuous Integration a. Is the practice of automating the integration of code changes from multiple contributors into a single software project. It's a primary DevOps best practice, allowing developers to frequently merge code changes into a central repository where builds and tests then run. Tools used - Jenkins, Gitlab CI, Circle CI are the most used ones.

3. Automated Testing a. The purpose of continuous integration is to automate the testing and building of the code. Whenever changes don’t break the functionality, this includes unit testing, integration tests, and end-to-end test. If you are in LabVIEW dev side, you can leverage unit testing libraries from VIPM like the caraya unit testing framework. LabVIEW dev world is actively using the unit testing frameworks to ensure code quality and that new changes don’t break the functionality. For text-based languages Selenium, JUnit, TestNG, Mocha, Jest, tools are most used for automated testing.

4. Containerization a. Containerization is an advanced topic but it’s at the roots of uninterrupted SW operation and user satisfaction. Dockers and Kubernetes are being used in most enterprise software solutions. They are used for reliability, redundancy, scalability, security, and self- healing.

5. Continues deployment. a. It’s for automatically deploying every change in the CI/CD pipeline for production environment.

6. Monitoring and Logging a. If you have SW that follows the CI/CD you need to also use observability tools to monitor the performance of your application and infrastructure to ensure they are functioning correctly. Most SW teams use Prometheus, Grafana, Splunk etc.

7. Documentation a. Documentation is critical for every stage of CI/CD pipeline, proper documentation is required for in the code and outside this is especially critical for maintenance, onboarding new team members, and compliance with various standards.

To Summarize as I said CI/CD is a broad are, and for engineering companies who are designing and developing systems like ATE, not all the steps are going to be needed but some of them are must to have like first 3 for any production level software you develop.

How to approach ML for the type of systems we design, develop & manufacture at EC

Data is the most valuable asset of 21^st century

For ML integration into your solutions, the most critical aspect is the Data. Your system should be designed to aggregate right data that you care about. In this domain everything starts with data and ends with data. Data is the most valuable asset of 21^st century and your systems should have way of aggregating data and pre-processing, in the other words, your SW should have ability to periodically harvest data, that you most care about to local repo or cloud for further processing. Even if you do not have a clear picture of what you are going to do with that data you need to have a way to aggregate. That’s a competitive advantage that any system should have, this day’s customers make decisions of procuring new systems heavily based on these types of abilities that are incorporated into your systems.

Data exploration

As you start collecting data and have some ideas on how to use it to improve your system performance or internal design process you need to get into the Data exploration stage. Data exploration is a very important step that is crucial step for any Data Science – Data Engineering - Machine learning tasks. Typically, data exploration involves your domain experts and ML or Data Science team into the conversation of what you need to get out of it. Domain experts help ML or Data Science team to understand the data, clean data and as a result they develop plans on how to approach the problem and after that they start the step called future engineering, they identify the features you need. For example, you need to prevent the SW crashes, they will explore and clean data and will identify the outliers and will indicate the event happening as result of crash and based on that your ML engineer will be able to select ML model that will try to predict the event before it happens. In some cases, Data Scientist will need to develop a one of a king model for most advanced problem detection.

ML Models

The world of ML engineering, there is approach of supervised and unsupervised models. “Supervised” model means that you have a data set that contains metadata with labels that indicate the certain conditions that you are interested in predicting. “Unsupervised” means that you do not have those labels in place and your model should have a way of identifying the abnormalities on its own and have an ability to self-correct and learn. To have an effective ML predictive model you must have training golden data sets to apply to model that will process your real time data and will give predictions based on it, to reach certain level of accuracy you need to continuously re-train your model. Ideally you need to have a dynamic training process in place which will continuously learn and improve prediction accuracy. You need to have dedicated ML engineers monitoring and implementing corrections as you move forward.

ML Tools and Models

ML industry has a variety of tools and processes. The barebow start for us is statistical models, like the Linear Regression, Logistic Regression, Support Vector Machine, Decision Trees, Random Forest, Gradient Boosting machines, Neural Networks. More advanced steps involve deep learning with neural networks. The most used tools and algorithms are available in Python. However, there are toolkits and model importers that can be used in LabVIEW for our solutions being deployed on the test stations or on edge devices. Some of the latest SoM devices have ML cores included in the HW layers and SW libraries are available as a part of Linux kernel. Those are for self-sufficient edge computing nodes that can make decisions on the edge and transfer data to centralized cloud or local servers for centralized data processing to incorporate AI and effective visualization.

For LabVIEW users. There are a few ways to leverage ML into the existing solutions

Python nodes in LabVIEW: You develop your Python script that does certain level of ML and integrate into your LV project using Python node available for Windows and soon there will be available versions for NI Linux target. In this case you can automatically supply the data sets for training to PY model and have it deeply integrated into your LV code base.

DeepLTK deep learning LabVIEW toolkit by Ngene:

Deep learning LabVIEW toolkit by Ngene has a set of deep learning models that you can use to train deep neural networks for solving various ML problems. Starting from time series data analysis all the way to dynamic predictive models.

NI Vision development module support TensorFlow and OpneVINO model import.

If you are developing ML models in TensorFLow or OpenVino you can bring pre-trained ML models to your LV project using the LV toolkit for model exporting. It works with the latest version of TensorFlow and OpenVino. In this case your model should be developed in TensorFlow or OpenVino and model should be frozen and exported to be imported and executed in your LV project using the LV vision toolkit. In LV project you need to provide input data which will be used with ready model to give ML driven insights into your LV code base for further automated action.

The LabVIEW Analytics and Machine Learning Toolkit: is a software add-on for LabVIEW that provides training machine learning models. You can use these models to discover patterns in large amounts of data with anomaly detection and classification, and clustering algorithms.

To summarize: As you can see there are various approaches of leveraging ML into your systems with LabVIEW and even more if you use text-based languages. However, everything starts and ends with the data if your systems are not aggregating data and you are not processing that data based on your needs this ML models not going to help you at alls.

Introducing a quality standard into an organization is not an easy thing. The difficulty comes from the fact that we are accustomed to a certain way of doing things, and changing habits and traditions is one of the most difficult things to do. Yet it is us who will have to change for a new standard to take hold. For us to make this change it is paramount that everyone in the organization deeply understands why this change is needed.

The need for the organization to comply to a certain internationally acceptable quality standard appears during a certain stage of organizational development. In other words, this need is a reflection of a certain change in the context of the organization. It is usually a stage when there are already many employees in the organization, when there appear new customers that are not familiar with the organization and that need assurance of the quality of its products, or when the projects are big and a certain assurance from the customer side is needed of the quality.

I will take the stakeholder triangle as the ground from which I will open up the essence of the ISO 9001-2015 (ISO) standard. Below is the stakeholder triangle. I will try to tie the main concepts of ISO such as processes, continuous improvement, nonconformity reports, … to the stakeholders. By doing so, I will try to induce the ISO from the stakeholder expectations.

Each stakeholder has a certain set of expectations. Below, I tried to list the expectations as they relate to the ISO standard. I will assume that each stakeholder has a purpose (a set of S-curves that it wants to climb). In other words, I will assume that the expectations are dynamically growing. I understand that each stakeholder has its own growth movement different from the other stakeholder, and as such, there is a certain balance that must be maintained for this triangle to work.

The concepts of the ISO 9001-2015 (in short ISO) are quite simple and intuitive. Hence, it is not difficult to comply with this standard. The difficulty comes from the need for getting a Certificate. And to get a certificate, we must invite in an auditor and prove to him that we are in compliance. But the only way to prove that we are in compliance is to have everything documented. The auditor understands only documents. And here is the difficulty. The difficulty is in introducing the habit of documenting to the organization – documenting everything. A good mind trick that could be deployed here is to imagine the company as a 100-person company, where not everybody knows everybody and that the communication between individuals must happen through documents, databases, and folders.

The main concepts of the ISO are the following:

1. Everything must be described as a process.

2. Everything must be approached by the PDCA cycle for both the product and process levels.

3. Incorporate Risk and Opportunity thinking in everything that we do.

Below, I make some comments for every concept.

1. Everything must be described as a process. This idea is encapsulated in Dr. Edward Demings quote: “If you can’t describe what you are doing as a process, then you don’t know what you are doing”.

   And these processes must be standardized and written down and placed in a place that is seen by everyone in the organization. Why standardized? Because it is through standardized processes that an organization can achieve predictability and repeatability throughout the organization. Imagine having a 100-person company and everyone does things differently. In this case, there is no organization. In this case all you have is just a group of people that have just come together. But there is no organization. Because in big part, the organization is its processes.

   The need for documented process becomes apparent as the company grows in numbers. Below is a picture that shows three different processes – individual processes, team processes, and organizational processes. The need for documenting arises in the third phase – organizational processes. Yet, since we have grown to be organizations through the first two phases, we still carry with us many of their processes.

Besides documenting the processes, the processes must have process criteria. This is needed to later improve on the process. If you don’t have criteria, then there is nothing to improve. To show the importance of this I will take one of the organizational processes – the process of converting the idea into a product.

Now, there are different ways of accomplishing this – some processes will be better than the other. For example, the below process certainly will need improvement. But this improvement can happen only if you know on what to improve, meaning – on the criteria of the processes that you have set (for example, speed, quality, resource, time, …). This adds quite a difficulty, because the criteria are mostly numeric, thus measuring these criteria becomes part of managing processes. And we are not good at it.

This also means that to change something, you must first change the process.

2. Everything must be approached by the PDCA cycle. The PDCA stands for Plan, Do, Check, Act. They must be implemented for both products and processes. The idea behind this approach is continuous improvements and innovation.

a. Check. For products, the check is the Test, the outcome of which is the Test Report with the Nonconformities Report. b. Act. Once we have the list of nonconformities, we can decide what to do – the Act. Here are the different possible Acts: i. Correction – this means to correct the nonconformities of the products. ii. Corrective Action – this means to correct the process that produced the nonconformities. iii. Improvement – this means to introduce improvements both on the product or the process. iv. Change – this means the change of the whole approach.

3. Risk & Opportunities Mindset. Risks and Opportunities must be looked at both separately and together. If taken separately, then risks are about potential losses, and opportunities are about a potential gain. But if taken together, you could think about the risks of taking or not taking an opportunity. Likewise, you could think about the opportunities that come from taking risks.

I will stop here. These are the points that I wanted to point out here. There are many other nuances in the ISO that will make this article much longer. I could also refer you to an article on ISO that appeared in EC Journal Issue 43 (2022).

Every audit is an opportunity to clean the house. Every audit is a check on the implementation of set processes, and their need for change. In short it is back to Continuous Improvement and Continuous Innovation of the organization itself. It is an exam that every department in the company and the organization as a whole must pass. And this exam is a yearly exam. We must pass it every year.

Last week (Jan 17-19) there was an expo in India - which Vigen and myself attended representing YEA Engineering in High-Tech Industry Ministry’s booth.

The expo took place in New Delhi - in a big convention center. The convention center had multiple buildings - the largest being the one with 5 Halls under one roof. All 5 Halls were occupied by Convergence India Expo.

To give a sense of scale - the size of the Expo hall was comparable to ~ x20 DigiTec areas. The content of the expo was also very similar to DigiTec. Everything related to Tech was represented but with some grouping by application. The content of the event was very diverse, therefore all the participants were presenting their services as generalized as possible.

The majority of the expo was occupied by either consumer electronics companies which produce in India (producing Speakers, Headphones, Smart Watches, Phone Chargers, USB cables, etc.); IoT companies presenting Enterprise Software Suites for digitization; or companies in Networking and Telecom - presenting switches, routers, access points and networking solutions. Many companies with similar products and services were there - from different states of India.

The companies producing Speakers had a smart strategy - playing loud Indian music through every speaker they possessed. Luckily, those companies were grouped together and were far from the Armenia’s booth.

The location of our Booth was chosen pretty well - it was on the first raw in the middle of the whole expo area (see in the map below)

It was strategically placed in front of the “VIP Lounge” - however we didn’t see much benefit from that position. In any case - there were a lot of people passing by and getting attracted by the Design of our booth to come near and see the content.

8 companies were chosen to be included in Armenia's booth, 7 showed up for the Expo. The 8th company members got COVID before their flight.

Participants with the Ambassador of the Armenia to India

YEA Engineering’s stand was on the front side of the Armenia’s booth. We had a Video-Clip showing our key solutions in Automotive and Semicon running in sequence.

We also had brochures of our key solutions - kindly printed and shipped to Delhi by Bhanu from Reyocto Infotech. Bhanu also joined us in Expo the 1st day.

Due to the diverse nature of the Expo - it was pretty hard to target visitors who would be interested in Automotive Test, Semiconductor Test, Electronics Test or general Engineering Services. One of us would stay near the booth and the other would roam around to find exhibitors who we can offer our services to.

Smaller part of the exhibitors presented Embedded System Design abilities and PCB Manufacturing and Assembly services. Many of the Embedded System Design companies were presenting similar products - all of them had GPS trackers for transportation, Connectivity devices for transportation and remotely controlled locks. These were the companies we spent most of our effort talking about potential Validation or Production test needs. Here we have a few leads to work on. We also spent time talking to Contract Manufacturers about their services - in search to find reliable CM to use for opportunities in the Indian Market - particularly for our High Power SMUs.

Over the course of 3 days there were a few visitors to our booth that seemed to be interested in our Engineering Services or Test Solutions - we’ll be working with gathered leads to generate opportunities. Probably the most interesting lead is a company called BOSON Motors (https://bosonmotors.com/) which is building autonomous vehicles for Farming applications and is interested in Test Systems both for Autonomy and for EV. Processing and categorizing the leads will take some time and will be a subject of another report.

The last day the whole Armenia delegation was invited to the Armenian Embassy. With the Ambassador we discussed the overall impressions, Vigen raised the question about high taxes for import to India. It was a nice and open discussion where the Ambassador and his staff would also share their impressions of India and tips on how to negotiate with local people and companies.

Overall impressions of the trip - this Expo was not just what we wanted and it would be better if the Ministry would have picked expo more aligned with our applications, but it was not bad either. There were some interesting leads worth working with. We found a few contract manufacturers worthy of assessment and a manufacturer of RF Absorbing foam that matches with ones we use form Emerson and Cuming.

Unfortunately other than Expo - this time there was no time for exploration of New Delhi or for some sightseeing. This time it was also cold - colder than in Yerevan. It was also unexpected for locals since none of the buildings were heated. The Air pollution was also extreme and the city was in smog during the whole daytime - that’s why all the airplanes would fly in and out of Delhi at nighttime. Needless to say how busy the Delhi airport can usually be - but this time around it was unexpectedly awfully busy and I almost missed my flight back home 🙂. And I would have surely missed it had I not ran dragging my suitcase through the

whole terminal to the Gate like I was in the Home Alone movie.

Some of the members of the Armenian delegation stayed on Saturday to visit Taj Mahal. Me and Vigen decided to leave it for the next time.