robot_api customized for the mobipick robot.
A pre- and concise Python API to control mobipick robot with simple commands.
Get a Mobipick Robot object using the robot's namespace:
import mobipick_api
mobipick = mobipick_api.Robot('mobipick')
Navigation:
# Get the robot's 2D pose using localization.
mobipick.base.get_2d_pose()
# Move the robot's base using move_base.
mobipick.base.move(21.0, 7.0, 3.141592)
Perception:
# perceive: the one perception call, a thin client of the mobipick_active_perception Perceive
# action (/mobipick/perceive), which does the work. Targets mix known objects (DOPE), open-set descriptions
# (Grounding DINO) and tables (table_<n>: cached poses on it are cleared, DOPE observes it).
# DOPE (pose selector) is active for the whole call. confidence (default "high") only matters
# for open-set targets: "low" = one Grounding DINO view + one VLM verdict, pose not committed;
# "high" = per target DISC alignment, Grounding DINO, close-up, Grounding DINO + VLM, committed pose.
# Returns one text: success|partial|failed, then one line per target.
text = mobipick.arm_cam.perceive(['table_2', 'multimeter', 'tennis ball'], confidence='high')
# partial:
# table_2: perceived, table observed with DOPE (confidence does not apply to tables)
# multimeter: perceived successfully with high confidence (DOPE; multimeter_1 at map (12.31, 3.02, 0.78))
# tennis ball: failed, VLM verification rejected all 2 candidate(s) (it saw: an orange)
# options (experimentation): use_vlm=False, align=False, observation_pose='observe100cm_right',
# dope_only=True or gd_only=True (not both), session_dir='~/sessions/x'
# Without the Perceive server the old closed-set observation below runs instead (deprecation
# warning); only targets that are pose selector ids (multimeter_1) can then be confirmed.
# legacy (deprecated, warns): DOPE for 5 s from the current arm pose or from each listed pose, no result
mobipick.arm_cam.perceive()
mobipick.arm_cam.perceive(observation_list=['observe100cm_right', 'observe100cm_front'])
# open-set detection with AnyGrasp's Grounding DINO + SAM2 (no grasping): accepted
# detections land in the pose selector as <class_id>_<n>; returns grasplan/DetectObjectsResult
# with 2D boxes/masks, map-frame oriented 3D boxes and the accepted objects, or None if unavailable
mobipick.arm_cam.detect_open_set('coke can', use_vlm_verifier=True, observation_pose='observe100cm_right')
# close-up open-set inspection at the table the robot stands at (mobipick_active_perception
# InspectObject action; perceive(..., confidence='high') uses it, call it directly only for testing):
# observation view, VLM triage, close-ups of what needs one, confirmed matches committed to the
# pose selector; with align=True and DISC running the base is first shifted along its heading so
# the observation view is centred on the object; use_vlm=False commits the Grounding DINO
# detections of one close-up instead. table only labels the recorded session.
# inspect_table(...) is the deprecated old name.
result = mobipick.arm_cam.inspect_object(['sugar box', 'coke can'], table='table_1', align=True)
result.accepted, result.poses, result.outcomes # e.g. ['sugar box_1'], map-frame PoseStamped per object, one line per query
# the building blocks it uses are clients too: proposals from the current view (nothing accepted)
# and one multi-image VLM verification per candidate (lists of detection ids)
first = mobipick.arm_cam.detect_proposals('coke can', max_proposals=5, observation_pose='observe100cm_right')
mobipick.arm_cam.verify_candidates('coke can', [[d.detection_id for d in first.detections]], commit=True)
# drive the base straight along its heading (robot_api Base.move_linear -> mobipick_base_motion MoveLinear; negative = backward)
mobipick.base.move_linear(20.0)
# query 6D pose estimate of a specific object
mobipick.arm_cam.get_object_pose('multimeter_1')
# query if a specific object was perceived or not
mobipick.arm_cam.is_object_inside_pose_selector('multimeter_1') # expected return value is a boolean
# remove pose-selector entries for objects whose semantic facts place them on a table
mobipick.arm_cam.clear_poses_for_table('table_1')
# match a map-frame pose to the nearest configured table center
mobipick.table_matcher.closest_table({'position': {'x': 12.1, 'y': 3.2, 'z': 0.8}})
The robot object can be created before the pose selector node is running. Perception
operations wait for the particular service they need and can be retried after a
startup-order or temporary service outage. If the service does not become ready,
the operation raises rospy.ServiceException with the unavailable service name and
retry guidance. The readiness timeout defaults to 2 seconds and can be configured
with the private ROS parameter ~pose_selector_service_timeout.
Manipulation (with MoveIt):
# move the robot's arm in configuration space to predefined semantic poses
mobipick.arm.move('transport')
# to see predefined semnatic poses do the following command in a terminal:
roscat mobipick_moveit_config mobipick.srdf.xacro | grep arm | grep state
# pick an object that was previously perceived
mobipick.arm.pick_object('multimeter_1', 'table_3', planning_scene_ignore_list=[], timeout=50.0)
# insert an object that was previously picked into a container, e.g. a box
mobipick.arm.insert_object('klt_3', observe_before_insert=False, timeout=50.0)
# assuming that mobipick has an object in its gripper, you can place it on a surface by doing:
mobipick.arm.place_object('table_3', observe_before_place=False, timeout=50.0)
mobipick_api depends and is inspired by robot_api which was developed by Alexander Sung alexander.sung@dfki.de
mobipick_api was developed and is maintained by Oscar Lima oscar.lima@dfki.de