PLEXUS
Project with Heureka for the new interactive science exhibition
project OVERVIEW
A visualization sculpture for ML interpretability to introduce the Artificial Neural Network to kids
Responsibility: User Research, Concept Design, Visualization, UX/UI Design, Electronics implementation
Keywords: AI, Visualization, UX Research, ML interpretability
Duration: 9 months
Sponsors: Heuraka the Finnish Science Center, Aalto University
background
The Product development Program of Aalto University, Finland has given a unique opportunity to bring students from Aalto and ISDI School of Design and Innovation, India together with Heureka to work on a project for the exhibition.The whole exhibition will contain approximately 10-15 single exhibits and it will be opened in April 2022. Therefore, Heureka challenges us to design a Artificial Neural Network sculpture that could be understood even by kids. Team Plexus presents a fully working Artificial Neural Network structure for Heureka’s forthcoming Intelligent Machines exhibition. Visitors can handwrite a number (0-9) on a screen in the input area, which is then recognised by ANN and displayed in the output area via three 28 X 28 LED panels. The LED panels will show the three most possible numbers in an interesting way. I present this study case with the purpose of exposing the adventures we have had during this entire year to achieve our final concept. Our activities started in September of 2020 in unique and unimaginable condition and concluded in May 2021 at the ADF Product Design Gala.
aim
The aim of the project is to develop a model that portrays the working of Artificial Neural Network
and display it in the form of a Kinetic sculpture.
decision making
At the start of the project, we were having two project options so the team along with the sponsors has discussed features and values that the projects must have. The pros and cons were discussed and listed and according to the group interest we decided to move ahead with ANN project. Decisions that concern the whole team for example team name, roles, distribution of responsibilities were discussed in team meetings and the final decision was made in a democratic way. Along with this the team had discussed and agreed upon a set of ethics, morals and team rules. Working groups were made to match the needs of the project. We have had 5 subteams and individuals were assigned to working groups based on skills of the individual and what skills they would like to learn. The project time-line and task assignments were done by the project manager. The working groups check the time-lines and tasks and get back to the project manager if any changes were needed. At the team meetings, every working group presents what they have been working during the week. In case there was a difference in opinions that is not being resolved or the team is split, the project manager took the final decision.
Communication Plan
Our 9 member team was split across 3 locations: Finland, India and China. After March 2021 onwards, we were only in Helsinki and India. The team plexus uses Telegram for communication because it is easy to communicate with global team as well. Doodle allows us to create polls that members can vote on to select meeting timeslot. Due to the time difference and schedule not every team member can join each meeting but then it is their responsibility to go through the meeting and discuss with other group members. If there are important decisions or schedules made, then the project manager contacts them and gets them on board. The sponsor joined us every Wednesday. All the team members were connected to the sponsor but the main point of contact was the project manager. The project manager was also responsible for participating on doodle and schedules with the PDP teaching team. We have had different groups with sponsorer on teams and there was communication between PM and sponsorer through whatsapp as well. Google Drive was used to store every document and access was given to all the team members as well as to the heureka. Using Microsoft Teams we could place video calls and can have live captions to understand better. Also, Mural has been used for ideation workshop.
Design Process
Research
Artificial Neural Network
Artificial neural network or simply neural network are computing systems inspired by biological neural networks that constitute brains. The neural networks are made of simple nodes which interact with each other to help in coming to a solution for a given task. These nodes loosely mimic the neurons in the brain. The connections between the neurons and the neurons contains weights which adjusts as the learning proceeds.
Typically, these neurons are grouped into multiple layers. Broadly, these layers can be classified into input, hidden and output layer. Input layer gets information from the input and acts as the entry point for the data into the network. Output layer is the layer where the result for the problem is seen. The layers in between the input and output layer are called the hidden layers. The picture on the top right corner summarises this same concept and is also the key idea which we want the kids to grasp.
User research
Our product needs to be easily understood and interacted with children so studying how children feel about our product is important and necessary.
In order to understand:
1. How the children’s understanding level of AI (Artificial Intelligence) & ANN (Artificial Neural Network)
2. Can children easily understand our simulation/low-fidelity product?
In February, we had an online interview with 5 children in a junior high school located in Tampere. In this interview, we got some feedback from them and introduced the basic principle of ANN.
We interacting with kids and get their opinions and perspectives on the future of computing.
Design Brief
Objectives
1. Build kinetic sculpture that explains ANN
2. Easy to understand especially for children
3. Easy to maintain and durable
4. Amaze, teach and entertain visitors about ANN
5. Interaction between visitor and network
6. From the front end to back end, displayed entire ANN process
Senario
Heureka, the Finnish Science Centre introduces the public to science and technology in an engaging and interactive way. Visitors can experience the joy of discovery through spectacular exhibitions, planetarium films, educational programmes and events. Located in the Tikkurila area of Vantaa, Heureka first opened its doors to the public in 1989. Heureka is one of Finland’s most popular recreational centres, attracting an average of nearly 300,000 visitors each year. Heureka’s mission is finding joy of discovery and experiences for everyone, committed to the most unique way to experience, learn and get excited about science. Their operations are subsidized by the Ministry of Education and Culture and the City of Vantaa.
Ideation
Reframing and Ideation Workshop
An ideation workshop is a dedicated session for coming up with new ideas. Unlike traditional brainstorming, ideation workshops are unique because they take place in unusual settings and introduce new stimuli. It is a small-group meeting format for proposing or solving problems in which participants are free to express their personal opinions and ideas.
There is an additive effect when people engage in thinking. When one person is struggling with a problem, many people sit together and discuss it, complementing and refining each other's ideas, creating a "resonance" of ideas, breaking through limitations with different ideas, and creating a "complementary effect". When one person puts forward an idea or a point of view, the idea or point of view is "resonant. Others are stimulated to respond with more ideas and opinions, and the ideas collide with each other to form a "mutual stimulation effect. This "complementary effect" and "mutual stimulation effect" makes the collective wisdom greater than the individual wisdom, and the ability to think creatively is enhanced. Because brainstorming is a collective and creative approach to problem-solving, interdisciplinary collaborators are more easily to produce a creative solution rather than a lone genius designer.
Brainstorming
Brainstorming is a method design teams use to generate ideas to solve clearly defined design problems. We were asked to identify customer needs and prioritizing customer needs, we write down as many as possible and found some interesting findings which are not be noticed during the previous discussion. Like hyper-causal interaction. We marked each need with numbers, and everyone can write down the number in the prioritizing order. This task is a warm-up exercises before we start the real ideation. The purpose of brainstorming, however, is to brainstorm, to stimulate inspiration, to explore ideas, and to some extent to allow for seemingly whimsical ideas, not to reach a unanimous conclusion.
Brainwrite
Brainwriting (sometimes called "individual brainstorming") is a method for quickly generating ideas by asking participants to write their ideas on paper (or online) rather than announcing them in tradional group brainstorming sessions. We conducted 6-3-5 brainwriting based on random brainstorm. By building new ideas on the ideas of others. Expanded on others' ideas and use theirs to stimulate my own. By swapping members' ideas with each other, the creative collisions and combinations can lead the discussion in unexpected directions. Many “wild” ideas present their real meaning when composed with other ideas. After 5 rounds, we got 39 expanded ideas.
SCAMPER
SCAMPER is actually an acronym and each letter stands for one thinking technique: Substitute, Combine, Adapt, Modify, Put to another use, Eliminate, Reverse. So SCAMPER is a collection of these various techniques and you can choose to use one or all of the seven methods to find solutions to your problems. Many thoughts discovered the others and the ego-self in the understate and process. Thus, no longer limit to the ego for the center of the narrow and small scope inside but jump to come into the extensive understanding realm those others together. Getting rid of the restraint of original thinking to evaluate the project itself.
Random
Word Association Also called “free association,” you draw random words from a bank. You then go around the group and invite team members to draw associations between those words at the topic at hand. In the process of Random Word Association, divergent thinking makes the conception smooth and flexible. During this process, we came up with 40 random word associations. We used the template to keep track of all the associations.The product manager emphasized that we shouldn’t imply that an idea won't work or that it has any negative side effects. Because there are no bad ideas in a brainstorming session, they all have the potential to be good ideas. It is important to distill as many ideas as possible in a given amount of time.
Final concept
After ideation, we started putting the layer system into one structure which will convey the basics of ANN. Journey from initial idea till final one is shown.
Ergonomics
Our final design is a result of a long journey. Our task was to mold a mathematical method into a kinetic sculpture. We were trying to understand artificial neural network in that level that it can be translated into a physical object. The main challenge was to make it clear and understandable not only for us but also for a curious ten year old kid who meets the technology for the first time. We were searching methods and logics where this complicated calculation system can visually act as a simple and straightforward interaction.
Our final concept displays the three main components of ANN. We can see the input layer where AI starts to analyse the given information, the hidden layer where it narrows down the possibilities, and the output layer where it displays the calculated results. And all these three layers can be seen at the same time above each other. The user gives the information - in our case draw a number from 0-10 - and can just follow the calculation with their eyes.
Develop
Prototype
Prototype is nothing but an early sample of the product build to test the concept or design. After finalizing our structure design, we started thinking about building the structure for the gala.
3D Printing
3D printing technology is used for making complex parts which are difficult to produce by conventional machining methods and it is very famous for making the first prototype to check the design aspects. As 3D printing is easily available in design factory we decided to check our design mainly curves by using the same. Using Cura software, we finalised the orientation and parameters for 3D printed. In this process, we realized that the curve is not as smooth as expected. So, we came up with new dimensions and finalized the gala structure.
Design
UX/UI Design
The ideation on the user interface started with the thought of delivering a fun, fascinating and futuristic experience to kids. The immediate question that arose was, whether it should use a stylus or a finger as interaction medium. We first started with understanding the implications of using either on the ANN network in the backend.
Once we found out that there is not much of a difference, the obvious choice was using the touch interaction since it will be more natural and will not involve the overhead of maintaining an extra device. With that out of the way, next we started to ideate on what components the user interface would contain.
We arrived at the conclusion that the user interface would be minimal with clear instructions so as to communicate effectively with kids.
The User Interface was to be divided into two main components, first being a division where the kids can draw the numbers and interact with the ANN at backend and the other division would contain the basic instruction about how the screen works and what the sculpture will accomplish after the user provides the input. Next we ideated on the buttons that the user interface would contain. Initially, we started out with the idea of including a clear and submit button. The thought was that on the press of the submit button the image would be sent to the backend for processing. On further discussions with Heureka we decided to keep the input dynamic i.e the input captured by the user would be interpreted in real time, eliminating the need for a submit button. Hence the conclusion was that the interface would contain only the clear button. Now with the foundational questions out the way, we proceeded with the fun part which is designing screens. We started with ideating visual for the screens, here is the first ideation from the same.
I created a wireframe based on the input website and visualized the overall flow. While making wireframes, I decided to make the order of priority out of the three features; I thought the input was the first one so I made the "Clear" button bigger and eliminate the rest of buttons, then changed the front page screen accordingly.
Shapes
Combination, subtraction and intersection of a few simple elements such as perfect circle and either squares or rectangles.For consistency I developed a few generic casing style depending on the size of window and type of the content. Generic frames are resizable rectangle frame with rounded corners. Button frames are always rounded corners. Same as the input area frame, buttons have some highlights. The shapes is the combination of square and circle. Slight gradients are applied on gold buttons vertically. Light spots in UI are used primally for two reasons. First, to call attention to important functionality such as input area and "clear" function. Secondly, to create mood to support thematic cohesion to high-tech but also kids-friendly such as input area frame, introduction frame and buttons.
Custom Colors
The input area and buttons is the canvas of the UI. The color scheme is heavily influnced by SRU color palette. The darkturquoise and deepskyblue are the main background color mest used and orange is accent color. The combination of dark blue and yellow creates harmony of cold and warm. In UI components certain colors are specifically codedoto represent certain function such as input area and confirmation.
Typography
Londrina Solid with drop shadow is uniformaly used in website UIs. Numbers and introduction text are very important data. They are rendered in bright white colors for best readability.
Next we started working on implementing the design. The languages we decided to implement this were Javascript, HTML and CSS. Since Javascript is the most popular language to build front ends and loads of libraries that can be used to implement designs. We first started out to build the the draw area using the HTML <canvas> element. Next we had to capture this image and send it to the backend for processing.
This is when we discovered the gradio library. Gradio is an open-source python library which allows you to quickly create easy to use, customizable UI components for your ML model, any API, or any arbitrary function in just a few lines of code. You can integrate the GUI directly into your Python notebook, or you can share the link to anyone. We downloaded the gradio code from GitHub and decided to make relevant changes in the course to suit our vision. It takes the input from the user interface, computes the probabilities of classes (in this case numbers from 0 to 9) based on the input (hand written number in the user interface) and tells what are the top 3 possible numbers for the given input.
Here is the final design after being implementing and customising the gradio library.
Information Design - Visualization
We built the electronics hardware for visualising data flowing through the neural network, and the final output. Each layer stores the data as a matrix, so the best way to represent those data is by making a matrix where each cell can be controlled easily. The main ideation round produced ideas involving hanging balls or bulbs, or single or multiple LEDs for each cell in the matrix. Later when the design for the structure is finalised, we decided to use LEDs to show the network data.
Information Design - Electronics
The neural network takes input as an image of size 28x28 pixels. The smallest grid which can visualize that data will have 784 LEDs. The hidden and the output layer data is also shown on similar sized grids. Controlling these many LEDs is not easy with traditional methods such as connecting them in rows and columns and sending signals to corresponding rows and columns. Discussions with the DF staff and the electronics team at Heureka led us to use addressable LEDs. Addressable LEDs offer a very efficient and easy way of controlling large number of LEDs.
Build
We have achieved this experience keeping the ergonomic requirements and needed dimensions that Heureka provided for us. The final structure fits into a 250 cm x 500 cm x 120 cm box and has a perfectly curved shape constructed with two ellipsoid. This method allowed us to make continuous smooth curves in this large scale including all the features and electronics hidden inside the structure. The input screen starts from 70 cm and has a gentle 20° angle in order to be usable for people of different heights. The inner metal structure is covered with thin black plexi to let the LED panels shine thus making the experience as immersive as possible.
Deliver
Evaluation
We would like to thank our sponsor representative Sampsa Piira and Vesa Lepistö who participated in the team’s activities, assisted in our work. Especially Sampsa help us evaluate the final sturcture in the field.
We show special gratitude towards offering their time and attention to host our team and co-operation throughout the project. Also, special thanks to the electronics team of Heureka for their kind help and being able to answer extensive questions and provide feedback time to time.
Deliver
what's next?
The sculpture has already been transferred to Heureka for the planning the new interactive science exhibition about artificial intelligence and robotics for its premises. The purpose of the sculpture is to amaze, teach and entertain visitors about artificial neural networks and how they work. Our intelligent structure is the last of intelligent technology exhibitions trilogy. Previous exhibitions were Intelligent Traffic and Intelligent City. It will be opened in April 2022.
What I have learned?
From learning to distribute amongst ourselves to interacting with colleagues from different streams. I also learned the basics of ANN and Machine learning and discovered various open source modules and components that were integrated with the final solution. My interpersonal skills have been improved, able to give clear and forthright feedback in a polite manner so as to not offend the other person. Considering the remote working situation we reliased that face to face meetings are more fruitful and impactful.