Skip to content

Data Design for Caesar RoME IIF

Jump to bottom
dehann edited this page Oct 9, 2019 · 44 revisions

Discussion here on which data will be supported and how, regarding performance and availability -- see reference discussions below.

VND, Small, Meta (softtype), big data

List of Important Data

Level 0 (aka skeleton): SkeletonDFG[Variable/Factor]

Level 1: DFG[Variable/Factor]Summary

Should be quickly searchable over thousands of nodes, possibly returned in summary

  • Position, Orientation [MAP_est? Also other related such as Velocity etc.]
    • get all poses close to coordinate [5, 3]
  • Type of MAP_est data [If 'MAP_est is just the numbers]
    • Only needed to make sense of MAP_est data -eg. MAP_est = [2,4,0.1] type = [x,y,theta] or Pose2
  • Time - Timestamp of the pose [UTC Epoch] (see Caesar.jl#269)
    • get all poses between time = [t1,t2]
  • Label [x1,x2,l1,etc.] (see Caesar.jl#251)
    • get x1
    • get x35 to x63
    • get all poses that saw landmark l1
  • The 'tags' [RobotId, SessionId, Variable, Factor, Landmark, Pose, ...]
    • get all Landmarks
    • get all for session_xyz
    • maybe if it is not important enough to want it searchable it should not be a tag
  • Naturally, all combinations of the above, eg. all the Poses close to time=t1 that 'saw' landmark l1.
  • 🔴 Variable softtype,

Level 2: DFG[Variable/Factor]

Still in graph, quickly accessible for use in calculations. Is this copied to the local (In Mememory) graphs for calculations as part of CSM.

  • VariableNodeData
    • Stores all values associated with inference
  • softtype and its meta data
    • Passed to user residual function for specialized computations there.
  • smalldata
    • Micro second timestamp, sensor timestamp used in calculations.

Level 3: <: AbstractBigDataEntry

Working so far: https://github.com/JuliaRobotics/DistributedFactorGraphs.jl/pull/131/files

Big Data, all data not stored in the graph itself. Various MIME or basic types (see Caesar.jl#274)

  • Image
  • Depth Image
  • Point Clouds
  • etc.

This looks like this in the code: https://github.com/JuliaRobotics/DistributedFactorGraphs.jl/blob/master/src/entities/DFGVariable.jl#L83-L96

Note the first-class citizens, and the abstraction of the solver data.

Potentially Separate Config Parameters

Currently undecided, and originally raised here (IncrementalInference.jl#143)

DFG usage

Note that for CloudGraphs DFG instances, we are going to make (userId, robotId, sessionId) required for a connection. So, user's will have to provide DB connection + scope of the instance (user, robot, session). This is needed to make sure the solver queries are efficient.

This will make multisession queries a special case, which must be addressed. Effectively, that means that we may need to make the functions more generic, but it won't change the data or affect efficiency, which is the current concern.

E.g. of creating a CloudGraphs-based connection and working in the graph:

# Create connection
cgDFG = CloudGraphsDFG("localhost", 7474, "neo4j", "test", #DB params
    "testUser", "testRobot", "testSession", # user/robot/session
    IncrementalInference.encodePackedType, # encoding/decoding
    IncrementalInference.getpackedtype,
    IncrementalInference.decodePackedType)

# Now can find :x0
exists(cgDFG, :x0)
# Or add a node
using RoME
addVariable!(cgDFG, :x0, Pose2)

Working Variable / Data Definition

See DistributedFactorGraphs.jl DFGVariable.jl

Legend

  • 🔴 new field ('redlines')

References

Clone this wiki locally