moebuta / moebuta Goto Github PK

construct a database to store the collected data with the good reasoning of [Lidar & Cam & Pos -> Planning / Actions].
use the data for few-shot inference when calling API.

validation agent to validate the accuracy and aligment

Motivation: Evaluate the effectiveness of the proposed framework

• prompt:
  • system instruction + real-time state info / changes + few-shot
• multi-modal:
  • cam + lidar + pos + historical state
• security:
  • w/ w/o threats
  • defence / error handling mechanism

Ablation study:

• lidar only / cam only -> plan/action
• target seen / unseen
• general prompt vs with restriction on prompt
• 0-shot vs few-shot

Metric:

• success rate of task completion
• cost of actions/tasks
• inference time of actions/tasks
• distance
• frequency of target lost after executing actions

Possible ways:

• Focus on the embodied agents:
  • w/o LLMs (rule-based, algorithms, learning-based), w/ LLMs with robotics:
    • Perception, Planning, Control
    • Problems: works for w/ LLMs still in the early stage, feels hard to categorise.
• Focus on the LLM:
  • works on LLMs: prompting, RAG, fine-tuning, RLHF.....
  • How have recent works applied these techniques when combining LLMs with robotics / embodied agents?
  • Problems: need to make associations with the embodied agent, feels hard to categorise.

survey paper

security on LLM robotics

rule-based constraint validation for safety control

validation agent for accuracy and aligment

Focus on evaluating the effectiveness of the whole framework against prompt injection attacks

in terms of ai agent:
we have user, cam, lidar, pos as perception, pretrained LLM as brain, command signals that execute mobile robot as action,

We generate perception interpretation, planning, and control together, but the performance seems not so good.

Alternative solutions:

• generate a general plan: e.g.: explore the area around (x, y), (x2, y2)
• then use a hard-coded algorithm / train a model to convert the plan into actions.

Security of AI agents in a broad aspect

CoreLocker and MInference are quite interesting. But how can I think of a topic with three objectives that can cover all of this stuff?

• obj1: explore threats and limitations of AI agent architectures (refer to PRDC)
• obj2: implement a novel secure method for AI agents in autonomous systems
• obj3: apply the concept into robotics realm for embodied AI agent
  • security stuff? should be focused on AI models
    • against attacks?
    • reliability?
    • performance?

Currently doing embodied AI agents found limitations:

• general pre-trained LLMs have limitations in handling specific tasks, even when few-shot prompting provided
• The zero-shot prompting-styled LLM module is vulnerable to prompt injection.

Options:

• fine-tune a small model for assistant purposes, input high-level instructions from LLM and output low-level control scripts
• RAG model for better few-shot prompting with GPT (prompt intensive)
• how can these be associated with security stuff?
  • prompt injection -> detection/prevention -> model
  • data poisoning

if focus on the security of embodied AI:
- prefer humanoid robots (trend)

• security topics?
  • prompt injections
  • data poisoning
  • network jamming
  • reverse engineering on the model (model inversion)
  • physical damage (embodied AI )