Robot Perception and Control

Logistics

Lectures

• Time: 00:00 - 00:00 (CST), MWF / TuTh
• Location: JBHT 000 (in person)

Office Hours

• Instructor: By appointment via email
• TA: 00:00 - 00:00 (CST), MWF

Logistics

Grading Policy

• HWs (30%): 5 home works
• Quizzes (10%): 5 quizzes each worth 20 points
• Midterm (30%):
• Final Project (30%): project report + presentation

A: 90% ~ , B: 80% ~ 90 %, C: 70% ~ 80 %, D: 60% ~ 70 %

Submission Policy

• HWs: penalty / day from the total points of the assignment after due.
• Final Project: No late submission.

Every submission is due midnight (11:59 pm) on the date specified.

What is Robot Learning?

Robot learning is a research field at the intersection of machine learning and robotics. It studies techniques allowing a robot to acquire novel skills or adapt to its environment through learning algorithms.

• Sensing: observe the physical world through multimodal senses
• Perception: acquiring knowledge from sensor data
• Action: act on the environment to execute task / acquire new observation

A key challenge in Robot Learning is to close the perception-action loop.

How to make robots learn?

These days, many robot learning methods are based on deep neural networks with various learning algorithms (supervised learning, unsupervised learning, reinforcement learning, etc.).

Basics

Backpropagation

Linear/Dense Layer

Convolution Layer

Recurrent Cells

Scaled Dot product Attention

An attention mechanism where the dot products are scaled down by .

• Motivated by the concern when the input is large, the softmax function may have an extremely small gradient, hard for efficient learning.

• Calculate similarity from and , and scale by the similarity.

• Can be viewed as differentiable dictionary.

Multi-Head Attention (MHA)

A module for attention mechanisms which runs through an attention mechanism several times in parallel.

• The multiple attention heads allows for attending to parts of the sequence differently.
• When , this is called self-attention.
• Only a small subset of heads appear to be important for the translation task. Especially the encoder self-attention heads, can be removed without seriously affecting performance [1].

Masked Multi-Head Attention

Masking of the unwanted tokens can be done by setting them to . The binary mask is added to the attention scores so that attention weight will be zero on those unwanted tokens.

Sample implementation:

qkv = to_qvk(x) # to_qvk = nn.Linear(dim, dim_head * heads * 3, bias=False)
q, k, v = tuple(rearrange(qkv, 'b t (d k h) -> k b h t d ', k=3, h=num_heads))
scaled_dot_prod = torch.einsum('b h i d , b h j d -> b h i j', q, k) * scale
scaled_dot_prod = scaled_dot_prod.masked_fill(mask, -np.inf)
attention = torch.softmax(scaled_dot_prod, dim=-1)
out = torch.einsum('b h i j , b h j d -> b h i d', attention, v)
out = rearrange(out, "b h t d -> b t (h d)")

Self-Attention vs Cross-Attention

Self-Attention: all of the and come from the same input source.

• Each position in the encoder can attend to all positions in the previous layer of the encoder.

Cross-Attention: the come from the reference source and the come from the querying source.

• This allows every position in the decoder to attend over all positions in the input sequence.
• One way to realize cross-modal fusion.

Inductive Bias

Inductive bias (IB) is the assumption(s) of data that the model holds [1].

• CNN: information in the data is locally aggregated. (strong IB)
• RNN: the data is highly correlated with the previous time. (strong IB)
• Self Attention: just correlating all features with each other. (weak IB)

Resources: Online Materials

• CS391R (UT Austin) by Yuke Zhu: lecture slides