Render one frame at a time

Hi,

Is it possible to render one frame at a time? Something similar to the following code?

# create world
world = dart.utils.SkelParser.readWorld("dart://sample/skel/cubes.skel")
world.setGravity([0, -9.81, 0])

# configure viewer
node = dart.gui.osg.WorldNode(world)
viewer = dart.gui.osg.Viewer()
viewer.addWorldNode(node)
viewer.setUpViewInWindow(0, 0, 640, 480)
viewer.setCameraHomePosition([0.8, 0.0, 0.8], [0, -0.25, 0], [0, 0.5, 0])

# run each step
for t in count():
    world.step()
    if t % T == 0:
        viewer.renderOneFrame()  # updateViewer() 

I don’t know of a way offhand. You’d have to look into the OpenSceneGraph API. Also since this is using dartpy, you’d have to make sure that OpenSceneGraph API is getting bound to python.

An alternative to consider is to record the simulation ahead of time, and then play it back step-by-step.

Or if you’re just trying to perform additional time steps per render cycle, you can use WorldNode::setNumStepsPerCycle(std::size_t).

Or if you want the world node to attempt real-time playback, you can create a RealTimeWorldNode instead of a regular WorldNode.

Thanks for your reply, however I don’t think the proposed solutions work in my case. I would like (among other things) to be able to load and remove dynamically objects from the world, and if possible decide dynamically when to render and hide the GUI. This dynamic behavior is one of the strengths of Python where I don’t have to close the program because one object was not placed as desired; I would just need to change its position at runtime.

one of the strengths of Python where I don’t have to close the program because one object was not placed as desired

I’m not sure what you mean by this. In C++ you can programmatically put objects into the world and take them out using world->addSkeleton(~) and world->removeSkeleton(~).

I think there may be an XY Problem happening right now.

Sorry, I meant that in Python you have an interpreter, you don’t need to recompile your code like in C++ if there is something that you don’t like.

So in the terminal, if you type:

$ ipython  # or 'python'

to open the Python interpreter, you can then:

import dartpy as dart

# create world and load an object
# check visually if the object is placed where you would like it to be
# If not placed where you want, you can directly move the object

# you can also write:
dart.dynamics.Skeleton?  # to get the documentation

# etc.

Basically, you can interact with the code as Python is an interpreted language.

I see what you mean now.

My reflex would be to run the viewer in a separate thread so you can keep the interpreter running, and then the rendering should just automatically update as you use the interpreter to move things in and out of the world.

Yep, I thought about that (using a Thread) but I wanted to make sure that there was not something already implemented in the C++ side (that at worst would just need to be wrapped in Python with pybind11).

Anyway, I just tried to use the threading library and also the multiprocessing library (because in Python, you don’t have per se real threads because of the GIL), but I couldn’t make the code works (using RealTimeWorldNode and viewer.run()). When using threads it still blocks the main thread, so I am unable to change the world, and when using multiprocessing it blocks or I get runtime errors like the following one:

node = dart.gui.osg.RealTimeWorldNode(world)
RuntimeError: Unable to cast from non-held to held instance (T& to Holder<T>) (compile in debug mode for type information)

Here is the Python file that I used:

import time
import dartpy as dart
import threading
from multiprocessing import Process, Value, Manager, Pipe

# create world
world = dart.simulation.World()
world.setGravity([0., -9.81, 0])

# create ground
urdfParser = dart.utils.DartLoader()
ground = urdfParser.parseSkeleton("dart://sample/urdf/KR5/ground.urdf")
world.addSkeleton(ground)

# create sphere
sphere = dart.dynamics.Skeleton(name="sphere")
shape = dart.dynamics.SphereShape(radius=0.1)
joint, body = sphere.createFreeJointAndBodyNodePair()
body.setMass(mass=1)
I = shape.computeInertia(mass=1)
body.setMomentOfInertia(Ixx=I[0, 0], Iyy=I[1, 1], Izz=I[2, 2], Ixy=I[0, 1], Ixz=I[0, 2], Iyz=I[1, 2])
shape_node = body.createShapeNode(shape)  # set shape
visual_aspect = shape_node.createVisualAspect()
visual_aspect.setColor([1, 0, 0, 0.5])
collision_aspect = shape_node.createCollisionAspect()
dynamics_aspect = shape_node.createDynamicsAspect()
joint.setPosition(index=4, position=1.5)
world.addSkeleton(sphere)

def thread_function(world):
    node = dart.gui.osg.RealTimeWorldNode(world)
    viewer = dart.gui.osg.Viewer()
    viewer.addWorldNode(node)
    viewer.setUpViewInWindow(0, 0, 640, 480)
    viewer.setCameraHomePosition([4., 4., 4.], [0, 0.25, 0.], [0, 0.8, 0.])
    viewer.run()

def function(conn):  # I tested several tools in the multiprocessing library
    world = conn['world']
    # node = conn.recv()
    # world = value.world
    # world = v['world']
    # node = dart.gui.osg.RealTimeWorldNode(world)
    node = dart.gui.osg.RealTimeWorldNode(world)
    viewer = dart.gui.osg.Viewer()
    viewer.addWorldNode(node)
    viewer.setUpViewInWindow(0, 0, 640, 480)
    viewer.setCameraHomePosition([4., 4., 4.], [0, 0.25, 0.], [0, 0.8, 0.])
    viewer.run()

# Thread
# t = threading.Thread(target=thread_function, args=(viewer,))
# t.start()

# Process
# v = Value('world', world)
m = Manager()
v = m.dict()
v['world'] = world
# parent_conn, child_conn = Pipe()
p = Process(target=function, args=(v,))
# parent_conn.send(node)
p.start()

# create box
box = dart.dynamics.Skeleton(name="box")
shape = dart.dynamics.BoxShape(size=[0.1, 0.1, 0.1])
joint, body = box.createFreeJointAndBodyNodePair()
body.setMass(mass=1)
I = shape.computeInertia(mass=1)
body.setMomentOfInertia(Ixx=I[0, 0], Iyy=I[1, 1], Izz=I[2, 2], Ixy=I[0, 1], Ixz=I[0, 2], Iyz=I[1, 2])
shape_node = body.createShapeNode(shape)  # set shape
visual_aspect = shape_node.createVisualAspect()
visual_aspect.setColor([1, 0, 0, 0.5])
collision_aspect = shape_node.createCollisionAspect()
dynamics_aspect = shape_node.createDynamicsAspect()
joint.setPosition(index=3, position=1.)
joint.setPosition(index=4, position=1.5)
world.addSkeleton(box)

time.sleep(10)
# t.join()
p.join()

I think that’s probably because you run p.join() at the bottom of this script. When you tell a thread handle to join, you’re telling the current thread of execution to wait until that other thread is finished.

Hopefully just removing that line should allow this script to be imported by an interpreter, and then you’ll have a separate thread running the GUI while the interpreter thread is still working.

Hi,

Sorry, I was not so explicit in the previous post. If you run the code, the box that should be loaded after the thread / process has started, is not loaded in the world; it blocks before creating it. The join() method is called at the end after it should create the box so I don’t think it has an impact.

One thing I’m noticing is it looks like the new box is spawned at a height of z=0, because I think only x and y are being set.

Is it plausible that the box is being spawned inside the ground object and then catapulted down to negative infinity?

For the simplest possible debugging, I would try putting a simple print('Made it here!') test right after p.start().

Hi,

In the original code I have several print to see where it got stuck. For most of them, it got stuck after p.start and before the box = dart.dynamics.Skeleton(name="box"). Like I initially wrote:

...
# start thread
t.start()

for i in range(100):
    print(i)  # last time, it only printed the first 10 numbers then got stuck...

# create box
box = dart.dynamics.Skeleton(name="box")
...

Anyway, instead of that, I looked at the C++ code, and wrapped the frame() method which is available from the OSG Viewer. This renders one frame at a time, the run() method is mostly equivalent to while (!viewer.done()) viewer.frame();. So now, I am just calling that method :slight_smile:

Thanks for your help!

That’s disappointing that the threading didn’t work. It would’ve been a really nice feature to have a live-updating scripting environment for DART. I’m glad you found a workaround, though!