I gave a talk a few years ago entitled Where Are the Robot Friends We Were Promised? The subject was robots in popular culture, think R2D2, and my delight in learning how to get us closer to having such a friend. I love robots, and if you need help exploring our relationship with machines, I am the droid you're looking for! Lately, I have been lucky to have some forays into robotics. Here is that story in pictures. Urs Fischer's PLAY - I worked with an excellent international team for a year to bring nine fully autonomous robots to life. If you swear to secrecy, I would happily talk for hours about the challenges. People's reactions to the chairs were, of course, the best part. For Puma, a line-following robot would help you train to race Usain Bolt. Another training robot I spent quality time with in a sweltering warehouse one summer was all about unique strafing controls. I contributed to multiple facets of these robot friends: hardware, firmware, vision systems, safety systems, and general team and client collaboration, often the most challenging aspect. It takes a lot of brilliant people to build the science fiction of our dreams.

Image descriptions generated by OpenAI's GPT-4.

Show image descriptions

GPT-4, the latest language model from OpenAI, is capable of describing images. This technology has the potential to make content more accessible. Check out how AI describes images below with minimal prompts. I only asked the algorithm to move past its limitations in describing humans by emphasizing the purpose was to aid the visually impaired.

Urs Fischer Play at Gagosian
Filming strafing on a prototype training robot
No items found.

A vast, minimalist interior space dominated by its expansive white walls and polished concrete flooring. The room is bathed in a cool, evenly distributed light emanating from rows of ceiling-mounted fixtures. The ceiling itself features a series of angled panels with clusters of lights that lend a modernistic touch to the room's design. Intriguingly, the room's emptiness is punctuated by three isolated items: an orange traffic cone standing near the center of the space, and two ergonomic office chairs on wheels, each positioned separately at different spots within the room. The chairs face different directions, adding an element of mystery to the otherwise stark setting. The sheer openness and the limited objects within the space evoke feelings of anticipation, as though the room awaits a transformative event or installation.

An expansive industrial warehouse exudes a raw, unrefined charm with its weathered concrete flooring and white brick walls that stand tall against the vastness of the space. Robust metal columns, painted in a muted blue hue, rise up to support a dark, exposed ceiling, interspersed with a few hanging light fixtures that cast a subdued glow.

Front and center, a sleek black tripod stands firmly on the ground, possibly set up for some recording or monitoring activity. A short distance away, a small robotic vehicle is positioned, with an orange cone next to it, indicating some form of testing or experimentation. Further back, another orange cone is spotted, suggesting a pattern or course laid out. In the distant corner, a solitary drone with glowing red lights hovers just above the ground, its presence hinting at technological undertakings within this setting.

This environment seems poised for innovation or research, with sparse equipment hinting at the cutting-edge activities that may transpire within its bounds.

Urs Fischer Play at Gagosian - Testing before opening
Play Interaction 1
Play Interaction 2
Play Interaction 3
Play Interaction 4
Play Entrance Display
Vision systems work
Puma Racing Robot 1
Line following computer vision tests
Puma Robot Commercial Shoot
Robot Friends
Eyeo 2019 Where are the robot friends we were promised?
No items found.

This is a panoramic view of a spacious room with reflective flooring. The walls are white, and the ceiling is high, giving the room an open, airy feeling. On the left side, there are a couple of colorful swivel chairs; one in magenta and another in light blue. A bit further to the right, a small group of individuals stands and interacts, appearing to be in conversation. Near the center, more individuals stand, with two in the forefront appearing to engage in a discussion. Further to the right, there's an isolated black swivel chair on the floor. The image captures the vastness of the space and the scattered arrangement of chairs and people within it.

A triptych, presenting three distinct frames within one composition:

1. Left Panel: A spacious gallery room with a reflective floor and a white wall. The leftmost side features a magenta swivel chair. Near the center, two individuals stand close together in conversation. Next to them is a yellow swivel chair.
2. Center Panel: This is a continuation from the left frame but zoomed slightly closer to the two individuals. They seem to be in the midst of a conversation or observing something. The magenta chair from the previous frame appears on the left, while the yellow chair is placed beside the individuals.
3. Right Panel: This frame is another continuation, presenting a portion of the gallery room. The two individuals remain the focal point, engrossed in their interaction. The magenta chair is on the far left, and the yellow chair is in proximity to the duo. The rightmost edge also suggests the presence of another person, although it's just a partial view.

Overall, the triptych captures a moment within a minimalist gallery space, focusing on a pair of individuals and their interaction amidst colorful chairs.

Triptych-style Image Descriptions:

Top Series: Child and Colorful Chairs Interaction

  • Frame 1: A bright gallery space highlights a young child in a green dress, her back facing us, approaching a pink swivel chair in the foreground.
  • Frame 2: The child, with tousled blonde hair, reaches out to touch the chair's backrest, her curiosity evident. In the background, an orange chair stands at a distance, seemingly observing the interaction.
  • Frame 3: With a mix of wonder and determination, the child interacts with the chair, as it appears to have moved. Behind her, multiple chairs including the distant orange one stand in contrast to the vast emptiness of the gallery.

Bottom Series: Boy Discovering the Chair's Movement

  • Frame 1: In the same gallery setting, a boy in casual attire cautiously approaches a black swivel chair. His body language suggests intrigue.
  • Frame 2: The boy, now closer, touches the chair with a bit more confidence. The surrounding area remains clear, keeping the focus on his discovery.
  • Frame 3: The final frame showcases the boy stepping back, creating some space between him and the chair, seemingly to observe or test the chair's response to his actions.

These images portray the innocent wonder of children discovering and engaging with interactive objects in a contemporary setting.

Triptych-style Image Descriptions:

Top Series: Dance with the Blue Chair
- Frame 1: In a spacious gallery setting, a person in a blue top and dark pants stands at a distance from a blue swivel chair, poised for some movement or interaction.
- Frame 2: The person steps closer to the chair, gesturing towards it as if beckoning or choreographing a dance.
- Frame 3: Engaging further, the person reaches out to touch or guide the chair, with a stance reminiscent of a dancer.

Bottom Series: Red Chair's Curious Observer
- Frame 1: A different individual, wearing a patterned shirt and white pants, investigates a vibrant red chair. The room is filled with bystanders in the background, adding a layer of social context.
- Frame 2: The individual is seen bending slightly, possibly trying to understand or manipulate the chair's movement. The red chair appears to lean away, creating an intriguing dynamic.
- Frame 3: There's a playful tug-of-war happening, with the individual pulling the chair while it resists. In the far background, a group of people observe the scene, creating a sense of public performance.

These sets of images showcase playful interactions between humans and animate chairs in an art gallery context, suggesting themes of curiosity, performance, and human-object relationships.

A triptych, presenting five distinct frames within one composition:

1. Initial Scene: In a plain room, a woman stands looking at a lone yellow autonomous chair.

2. Interaction Begins: The woman approaches and touches the chair's backrest, initiating a connection.

3. Attempt to Guide: Stepping back, the woman raises her hand as if signaling the chair to follow her.

4. Response from Chair: The chair tilts forward, appearing to react to her gesture, suggesting it's ready to move.

5. Paused Movement: As the woman briefly stops, so does the chair, waiting for her next cue.

6. Concluding Scene: With a sense of accomplishment, the woman confidently walks ahead, and the chair follows her closely.

The sequence portrays a woman experimenting with and successfully guiding an autonomous chair in a spacious room.

The image displays a WARNING sign with the following details:

- Main Warning: Autonomous moving vehicles in action, and there's a possibility that collisions may occur.

- Do not sit on chairs: A list of precautions is mentioned:
 1. Do not leave children unattended.
 2. Do not touch chairs/vehicles.
 3. Do not bend over the chair.
 4. Keep your head away from the vehicle.
 5. Do not put personal items on the floor.

- Visitors: They are advised to remain aware of their surroundings at all times.

- Danger: Vehicles are powered by lithium-ion batteries.

- In case of fire or smoke: People are directed to use the designated fire exit.

- The bottom of the sign emphasizes: Enter at your own risk.

This sign seems to be in a location where there are autonomous moving vehicles or chairs, perhaps for a display, show, or experiment. It advises visitors to take precautions and be aware of the potential dangers while in the vicinity.

The image showcases a computer interface related to development and visualization for depth cameras. The main window on the right appears to display a 3D point cloud visualization, with various shades representing depth. The point cloud seems to depict an environment with objects scattered around, distinguishable by different depths and colors. On the left, there are multiple panels detailing settings, options, and properties, including camera settings and visualization options. Several error messages are also present at the bottom of the screen, indicating some connectivity or processing issues. Overall, the interface appears to be a development and diagnostic tool for working with depth camera data.

The image displays an outdoor running track during nighttime or low-light conditions. The track is composed of red-brown rubber and is segmented by white curved lines, indicating separate lanes. In the foreground, a line-following robot, which might initially be mistaken for a bag or equipment due to its compact size, is positioned on the track. Further in the distance, a person wearing a black outfit with a bright-colored waistband, possibly a runner, is captured mid-stride, giving a sense of motion. Yellow markers can be seen on the track, likely indicating distances or relay handoff points. The perimeter of the track is surrounded by a dark wall or fence, and there are dimly lit lights in the background.

The image shows a computer screen displaying multiple elements:

1. XQuartz Window: In the upper left corner, we can see a window titled "XQuartz," which is a software package that provides the X Window System for macOS.

2. Pygame Window: Beneath the XQuartz title bar is a window labeled "pygame window." This window displays what appears to be a road or pathway, with a portion of the road highlighted in a red rectangle and another portion highlighted in a green rectangle. These rectangles may represent regions of interest (ROIs) being processed or analyzed.

3. Code Editor: On the right-hand side of the screen, there's an open code editor displaying Python code. The visible code snippets seem related to computer vision, especially with the use of the OpenCV library (indicated by the "cv2." prefix). Some operations and functions visible in the code include:
   - Image thresholding (`threshold the image`)
   - Color conversion (`cv2.cvtColor`)
   - Gaussian blurring (`cv2.GaussianBlur`)
   - Histogram calculations (`cv2.calcHist`)
   - Histogram normalization (`cv2.normalize`)
   - Setting up a termination criteria (`Setup the termination criteria`)
   - Initialization of mean shift (`meanshift is initialized`)

From the code and the visual representations on the screen, it seems the user is working on some sort of line-following or object tracking algorithm, possibly for a robot or autonomous vehicle, using the OpenCV library for image processing tasks.

Editor's note: ChatGPT won't describe humans and was unable to help with this one. In the above instances I asked it to anonymize the humans, but in this case I will say image is of Usain Bolt seated next to the Puma Beatbot.

The images display a room from different angles. The room appears to be an office or a workspace with a desk, computer, and various items scattered around. The robot showcased in the center and circled in the images seems to have two large "eyes", which are possibly LED lights or screens. Editors note, this is from my talk and I am highlighting an interaction with Vector, a personal robot by Anki.

This appears to be a scene from an event or presentation. There's a projection on the screen showing three identical images of a robot, which seems to resemble the robot from the previous image. There's also a person on stage near a lectern with a logo "eyeO". The atmosphere suggests it's a tech or innovation-focused event. Editor's note: Where are the robots friends we were promised? Lightning Talk at EYEO Festival in 2019