As part of Telexistence’s effort to find the best talent from anywhere on the planet, we work with Master’s and PhD interns throughout the year. Sophia connected with us via the MIT-Japan Program and decided to relocate to Japan for six months to work together.

Background: MIT Master’s of Engineering Degree in Mechanical Engineering, MIT Bachelor’s of Science in Mechanical Engineering. Professional Interest and Experience in Product Design and Development.

Internship Period: August 2023 – February 2024

Team: Hardware Engineering, Retail Robotics

What did you pursue in your Master’s research, and how does it fit with what we do at TX?
I finished my Master’s degree in Mechanical Engineering, under the Advanced Manufacturing and Design program at the Massachusetts Institute of Technology. Under this program, I undertook coursework in product design, mechanical design and supply chain/operations management. For my thesis work, I collaborated with Texas Instruments, a semiconductor manufacturer, to develop tools – including machine learning models, hardware, and software – to improve semiconductor yield through better wafer defect detection.

I think a lot of the skills I’ve learned through my Master’s program and thesis project relate directly to projects I have been assigned at TX. On the design side, I’ve been tasked with mechanism design, test bench design, and conducting material characterization studies. Working at TX has allowed me to extend my abilities in designing for real world manufacturing standards and has helped me become even more familiar with the CAD software needed to achieve this.

Unexpectedly, I’ve also happened across a project where I get to utilize my manufacturing supply chain/inventory management knowledge through developing models for store operation. I think the skills I’ve learned and practiced at TX will most definitely relate to my future career as an engineer.

What is your biggest achievement or learning through this internship so far?
Of course, this internship has allowed me to refine my technical skills in mechanical engineering and design. But one of the biggest skill growth I’ve noticed in myself is cross-functional collaboration and cross-cultural communication. Being able to interact with Japanese vendors and experts has allowed me to develop my communication skills in Japanese, specifically the vocabulary/language needed for mechanical engineering/design engineers. In addition, on a broader note, this experience has taught me how to generally conduct business in Japan, which I think will be helpful in the future. I hope to work in the product design and manufacturing realm, and of course manufacturing technologies will continue to involve Japanese companies, so having this skill set will be an advantage in future collaborations.

What are the three words that best describe the TX team or culture?

Flexible: The company culture is incredibly flexible. There are no hard requirements regarding time to report to the office, or remote/hybrid/in-person, as long as the work is done and goals are achieved. This allows many of the employees to work at a time that is best for their lifestyle. In addition, the start-up phase of the company means the directives by project can vary quite a lot from week to week, sometimes the vision for a project can pivot; the team is really good at adjusting the workflow and being flexible to match the direction of the vision.

Goal-Oriented: This type of mindset reduces time wasted – the priority is that objectives are achieved and the milestones are met, so employees are free to work in the method that works best for them. This means there can be a reduction in time spent doing unnecessary tasks and red-tape/bureaucracy.

Diverse: The company is made up of many employees from different nationalities and backgrounds. It works well since it offers a well rounded team with many different problem-solving methods, and it is interesting to get to know each others’ previous experiences and learn from others.

Would you recommend TX as a place to have an internship and why?
Absolutely. It is rare to come across such a flexible and encouraging work environment, and as an intern, this means that you can pick and choose projects/tasks that best align with your goals and professional interests. Since the team is small, many of the tasks we work on have a direct impact on the actual deployed robot. The team is incredibly helpful and willing to share their expertise, so I never feel like I am asking too many questions, meaning I’m learning new things everyday.

As part of Telexistence’s effort to find the best talent from anywhere on the planet, we work with Master’s and PhD interns throughout the year. Shreyas reached out to us in June 2022 showing his strong interest in the opportunity to work together. It took us only a couple of weeks to agree that it would be a great fit and to sign the internship agreement.

What do you pursue in your Master’s, and how does it fit with what we do at TX?

I am a Master’s Student at the Graduate School of Engineering of Hokkaido University. I am working at the Smart Structures and Systems Laboratory in the Division of Human Mechanical System Designs and my research is focused on developing Compliant Grippers for robotics using SMA actuators. While developing the experimental system for my research I have come to realize that I have a large
knowledge gap specifically related to the low-level software development of robotics systems. For example, the SMA (Shape Memory Alloys) Actuators are really difficult to work with because right now there are not many reliable control methods for the systems, and in most cases, we have to develop our own low-level codes for it to work. I explained my situation to my manager during my interview and he explained that it would be a great opportunity for me to work on similar problem at TX. Converting this problem into a mutually beneficial opportunity I have been working with the Mechatronics Development Team at TX to develop low level codes that would eventually make the inter-modular communication much easier inside the robot. Since coming to TX, I have noticed an exponential growth in my knowledge of low-level software development, and that is most definitely going to help me a lot with my research.

What is your biggest achievement or learning through this internship so far? What challenges have you faced while working at TX and how did you overcome them?

I have yet to figure out my biggest achievement at TX as I am constantly working on better things every day and it gives me a sense of accomplishment every time! However, I do know that my biggest learning is not something Technical rather a soft skill. I have some work experience prior to shifting to higher studies and during that time I was under the impression that the idea of leadership and teamwork was directly related to one’s intrinsic ability to put up a façade while communicating with people. After coming to TX, I have realized that teamwork and leadership, in fact, need real personalities and that in turn makes communication much easier. I have learnt a lot from my team here at TX and I intend to use these learning all my life!
Coming to the challenges I have faced. I understand that I have a huge knowledge gap on the software side of robotics. While giving the interview I had communicated that to my manager with a lot of emphasis. As a result, he asked me to start working on low-level codes for the TX robot. This was definitely challenging to me. However, the way it turns out I started bombarding my colleagues with a million questions. One might expect the people to get tired of answering so many questions, instead I received a million answers in return. That built a very open and inclusive
environment which allowed me to try out different things with all the freedom without worrying about making a mistake. It never felt like I was working “for” someone, rather it was like we were all trying to come up with a solution together. That, in my honest opinion, is the definition of a healthy work
environment. And that is a basic requirement to overcome any challenges.

What are the three words that best describe TX team or culture?

Progressive:
I have been following TX’s progress over the years and it never fails to impress me. I was really awestruck by the videos they upload on YouTube. You could almost take me for a die-hard fan! I always wondered is it just for show or the company actually has a progressive mind set when it comes to
Research and Development in Robotics. Having worked in other robotics Start-ups I have built a keen and sceptic understanding. But when I actually saw the CTO, the CEO, Head of Mechatronics Engineering and all the other employees I realized that it was truly, in every sense of the word, Progressive.
Inclusive:
As I explained before the people at TX are really open to questions and always ready for a conversation be it work or play. That makes for a really inclusive environment.
Incredible:
Well, this could be a biased opinion but I think TX is an incredible company that is working on state-of-the-art technology for a very difficult problem statement.

Would you recommend TX as a place to have an internship and why?

I would wholeheartedly recommend TX as a place for internship. The people here are very encouraging and never back down when you bombard them with questions, which is one of my bad habits. They make you feel involved and you get real hands-on experience which is really rare these days. I think for people who believe in taking the initiative and working on challenging problems TX is a very good place to start. This company can provide hands on experience over a wide range of technologies that are sometimes hidden away in research papers. It is really interesting to see the constant iterations of prototyping and product development cycle produce great Robots. Join TX and have fun while gaining tons of experience!

Telexistence Inc. today announced the mass production of its originally developed artificial intelligence robot, TX SCARA, to be installed in 300 FamilyMart stores, Japan’s top-tier convenience store chain in major metropolitan areas, starting later this month, validating its AI-based “robot-as-a-service” solution for grocery retailers.

TX SCARA was created to do the specific task of restocking refrigerated shelves with bottles and cans, a repetitive, tedious job generally performed by employees in often uncomfortable settings. TX SCARA can be in operation 24/7, replenishing shelves at a pace of up to 1,000 bottles and cans per day, relying almost completely on its AI system (known as “GORDON”) to know when and where products need to be placed on the shelves. 

The implementation of AI robots in FamilyMart stores will allow retailers to take advantage of the newly created time and economic “surplus” in the store environment. Retailers can focus on further improvements in the store environment for both employees and customers, as well as the profitability of each store.

Tomohiro Kano, General Manager of Store Development Department and Railway and Corporate Franchisee Department, of FamilyMart, commented, “The decline in Japan’s labor population is one of the key management issues for FamilyMart to continue stable store operations. The introduction of Telexistence robots into FamilyMart stores will eliminate the need for store employees to replenish beverages in refrigerators, and the newly created time can be reallocated to customer service and shop floor enhancement, leading to higher quality store operations.”

Kano concluded, “We will continue to work with Telexistence to create a new way to operate stores.”

This initiative is a small step toward realizing Telexistence’s mission of improving all simple labor workflows in human society with robots. Telexistence is promoting this initiative based on a collaboration with NVIDIA and Microsoft Japan. Specifically, NVIDIA GPU-accelerated AI technologies are embedded in hardware originally developed by Telexistence to provide AI and remote control of robots via Microsoft Azure platform.

“Currently, the greatest limiting factor for any industry, in any nation, is human resources. Especially in developed countries like Japan, the shortage of human labor is rapidly becoming apparent in the retail and logistics industries, which support human life in society,” said Jin Tomioka, CEO of Telexistence. “We are promoting large-scale production and social implementation of our robots for industries other than factory automation, industries that function as the infrastructure for daily life in society. We also will bring our automated labor solutions to other regions, such as the United States.” Tomioka added, “Telexistence’s robots will become a powerful influence, providing a virtually inexhaustible supply of new labor. The new surplus resources will be appropriately redistributed through market mechanisms, and in the long run, this can lead to the transition of human society to be significantly augmented by automated labor.”

TX SCARA is autonomous with GORDON during normal operation. In rare cases where the AI encounters issues due to miscalculation or external factors such as an item falling or being out of place, remotely located Telexistence employees using VR glasses take control of the robot to correct issues across stores. Each TX SCARA can replace one to three hours every dayof human work at a single store.

The market potential for TX SCARA is vast. In Japan alone, there are 16,000 FamilyMart convenience stores, plus approximately 40,000 more from other convenience store chains. Telexistence next plans to expand to convenience stores in the U.S., where more than half of consumers say they visit one of the country’s 150,000 convenience stores at least once a month.

“Moving from proof-of-concept to mass production of 300 remotely-operated robots with deep AI capabilities is a major step forward,” notes Dr. Lewis Pinault, Airbus Ventures Partner based in Tokyo. “As an early investor of Telexistence, we are thrilled to see the company’s diverse, richly talented team begin to take flight, rolling out their advanced technology across Japan and activating new channels in support of their future expansion in North America.”

Masataka Osaki, Japan Country Manager & VP of Corporate Sales, NVIDA:
“Entire industries are transforming as customers build their next generation of robotics and edge AI applications on NVIDIA platforms. Aiming to empower retailers to address labor shortages, Telexistence’s robotics solution, powered by NVIDIA’s GPU-accelerated technologies, will enhance experiences for both retail customers and employees.”

Tomoko Mikami, Managing Executive Officer, Small Medium & Corporate Business, Microsoft Japan Co., Ltd.:
“Microsoft Japan will be collaborating globally with Telexistence in terms of cloud platform and business development in the implementation of Artificial Intelligent Robotics for qualitative improvement of industry, as promoted by Telexistence. We are pleased that our platform can contribute to this project, and Microsoft will continue to support the project to make a significant impact not only in the Japanese market but also on a global scale.”

---

<Technical Collaborator Details>

Main initiatives powered by NVIDIA

■ Improvement of TX SCARA’s eye and brain functions utilizing NVIDIA Jetson edge AI platform

TX SCARA is powered with the NVIDIA Jetson TX2 module on the head unit which is utilized to transmit video inputted from the camera. Furthermore, the NVIDIA Jetson AGX Xavier module installed at the foot unit attains autonomous operation functions using AI. For example, the TX SCARA is designed with a strong arm and a camera angle so that the same robot hand can grab all types and shapes of beverages sold at convenience stores. Furthermore, NVIDIA Jetson AGX Xavier improves functionality by running image recognition neural networks in parallel, such as recognizing optimal gripping points that differ from beverage to beverage.

■ NVIDIA DGX Station improves recognition accuracy of GORDON

TX SCARA monitors beverage inventory availability 24 hours a day. While receiving this, the AI system “GORDON” predicts and schedules product-display timing and instructs TX SCARA to refill beverage stock. NVIDIA DGX Station, an AI workstation, was adopted as the training environment for GORDON. By simulating product displays in a variety of patterns, it can create an efficient and optimum image recognition model.

■ Smooth remote operation and prevention of VR sickness

If the AI display fails because of an unanticipated environmental change, TX SCARA will shift to Telexistence mode (remote control) and a remote-control operator will make corrections via VR operation. “VR sickness” can occur when there is a misalignment between vision and body senses because of delay in 3D video transmission. In order to prevent this, the system uses NVIDIA’s powerful GPU-accelerated compute capabilities for advanced image processing performance, which enables video transmission at a maximum speed of 50 milliseconds (*).

*: The amount of time (including network delay) that elapses between when the video is inputted into the robot camera and when it is displayed at the operator’s display. A “millisecond” is a thousandth of a second.

---

Main initiatives powered by Microsoft Japan

TX has selected Microsoft Azure as the cloud platform for the robot solution service by TX SCARA and is using it for the robot’s task management system.

■ Microsoft Azure used by GORDON to make task decisions

The inventory information scanned by TX SCARA, which monitors beverage inventory 24 hours a day, is captured by Azure’s cloud system as digital twin information, and the AI system GORDON uses this information to predict the optimal beverage display based on past sales trends at each store and demand by time of day. The AI system “GORDON” then predicts the optimal beverage display based on past sales trends and demand by time of day and determines the next task (*) to be executed by TX SCARA and give instructions.

*: A task is a display task such as “Pick up products from point A and place them at point B.”

■ Database to Minimize Out-of-Stock Merchandise

TX SCARA’s cloud system by Microsoft Azure maintains and manages a database (*) of sales trends at each convenience store where TX SCARA operates. This enables TX SCARA to predict and execute optimal display tasks calculated from current inventory information scanned by TX SCARA in real time and accumulated historical sales trends, thereby minimizing out-of-stocks in stores.

*: Data on products displayed, date and time of display, and number of units.

■ Collaboration in the U.S. Market Entry

TX and Microsoft will strengthen our cooperation in business development overseas, especially in the U.S. market, by leveraging Microsoft’s global corporate customer network and its global startup support program.

---

Telexistence is strengthening its R&D, Business Operations, and Corporate organizations to move forward at an accelerated pace as the business enters a growth stage. Please see below for current open positions.

• Head of Mechanical Engineering
• Mechanical Engineer
• Robotics Automation Perception Engineer
• Robot Maintenance Engineer
• Robot Systems Engineer (Deployment Engineer)
• Robot VR Operator
• Finance Lead
• Accounting Manager
• Sourcing & Procurement Manager

See Careers Page for more details about each position.

---

See Telexistence Press Kit for more photos.