Skip to content

Data Design for Caesar RoME IIF

Jump to bottom
dehann edited this page Oct 14, 2019 · 44 revisions

Discussion here on which data will be supported and how, regarding performance and availability -- see reference discussions below.

VND, Small, Meta (softtype), big data

List of Important Data

Level 0 (aka skeleton): SkeletonDFG[Variable/Factor]

Level 1: DFG[Variable/Factor]Summary

Should be quickly searchable over thousands of nodes, possibly returned in summary

  • Parametric Point Estimate (PPE) are convenience byproducts of data full marginal estimation:
    • Only needed to make sense of max/mean/modefit/... data -eg. max = [2,4,0.1]
    • softype describes format, :Pose2 means [x,y,theta]
    • Also please define the fields somehow (maybe in documentation), for example :max will be there and the name won't change to something else eg. :maximum DFG #158.
    • [If 'MAP_est is just the numbers]
  • Time - Timestamp of the pose [UTC Epoch] (see Caesar.jl#269)
    • get all poses between time = [t1,t2]
  • Label [x1,x2,l1,etc.] (see Caesar.jl#251)
    • get x1
    • get x35 to x63
    • get all poses that saw landmark l1
  • The 'tags' [RobotId, SessionId, Variable, Factor, Landmark, Pose, ...]
    • get all Landmarks
    • get all for session_xyz
    • maybe if it is not important enough to want it searchable it should not be a tag
  • 🔴 softtype name as ::Symbol,
  • Position, Orientation [MAP_est? Also other related such as Velocity etc.]
    • get all poses close to coordinate [5, 3]
  • Naturally, all combinations of the above, eg. all the Poses close to time=t1 that 'saw' landmark l1.

Level 2: DFG[Variable/Factor]

Still in graph, quickly accessible for use in calculations. Is this copied to the local (In Mememory) graphs for calculations as part of CSM.

  • VariableNodeData
    • Stores all values associated with inference
  • softtype and its meta data
    • Passed to user residual function for specialized computations there.
  • smalldata
    • Micro second timestamp, sensor timestamp used in calculations.

Level 3: <: AbstractBigDataEntry

🔴 Working so far: https://github.com/JuliaRobotics/DistributedFactorGraphs.jl/pull/131/files

Big Data, all data not stored in the graph itself. Various MIME or basic types (see Caesar.jl#274)

  • Image
  • Depth Image
  • Point Clouds
  • etc.

This looks like this in the code: https://github.com/JuliaRobotics/DistributedFactorGraphs.jl/blob/master/src/entities/DFGVariable.jl#L83-L96

Note the first-class citizens, and the abstraction of the solver data.

Potentially Separate Config Parameters

Currently undecided, and originally raised here (IncrementalInference.jl#143)

DFG usage

Note that for CloudGraphs DFG instances, we are going to make (userId, robotId, sessionId) required for a connection. So, user's will have to provide DB connection + scope of the instance (user, robot, session). This is needed to make sure the solver queries are efficient.

This will make multisession queries a special case, which must be addressed. Effectively, that means that we may need to make the functions more generic, but it won't change the data or affect efficiency, which is the current concern.

E.g. of creating a CloudGraphs-based connection and working in the graph:

# Create connection
cgDFG = CloudGraphsDFG("localhost", 7474, "neo4j", "test", #DB params
    "testUser", "testRobot", "testSession", # user/robot/session
    IncrementalInference.encodePackedType, # encoding/decoding
    IncrementalInference.getpackedtype,
    IncrementalInference.decodePackedType)

# Now can find :x0
exists(cgDFG, :x0)
# Or add a node
using RoME
addVariable!(cgDFG, :x0, Pose2)

Working Variable / Data Definition

See DistributedFactorGraphs.jl DFGVariable.jl

Legend

  • 🔴 new field ('redlines')

References

Clone this wiki locally