With more and more Machine Learning 3D applications that are relevant to content creation within VFX, Animation, Games and Film popping up by the minute, I really wanted to get into it and see how I can use it in my daily work. Maybe let a ML network tweak my simulation parameters to match a reference? Maybe take over parts of simulation calculations, that take a shitton of time to calculate the “normal” way?

All of them were quite far fetched. But some of the “far-fetchedness” was due to a big barrier to actually get to work even.
First off Machine Learning itsself has a steep learning curve. Just the math part is not to be unterestimated and not to mention all the different kinds of Machine Learning processes, jargon, model types etc. And it’s evolving faster than most other areas in computer science.
Additionally Machine Learning frameworks in Python usually come with quite a hefty dependency list. Managing them yourself is near impossible, so you have to rely on venv+pip or conda. Which both already have a learning curve themselves.
After that you still have to learn a deep learning framework like Tensorflow, PyTorch or Keras and get to work.

And after all that you still need to design your model and where you can get your data from.

Not all that easy.

But not all of the mentioned is actually necessary. Machine learning frameworks are not essential in creating machine learning models. They are designed to help, but are therefore not necessary. At least not when dealing with the simplest tasks.

And exactly for that reason I wanted to see if I can create the “simplest” ML task inside Houdini. But with the twist of also creating my own data instead of using MNIST.

Goals

With the project I wanted to cover the whole ML pipeline and do everything inside Houdini.

• Data generation
• Model Training
• Evaluation/Application

Additionally I wanted to focus on not relying on any external files or environments. Everything should be able to play directly with just the hipfile.

Solutions

• Data generation and model training is done in PDG
• Evaluation is done in SOPs
• The model is designed in numpy without any external libraries

And instead of relying on external python scripts that get imported, all code is written insdie of a multiline string parameter that gets referenced around. Actually very similar to #include in C like languages.

Result

The result let’s you do every step very easily and it’s very quick to execute. After roughly 5 minutes all data has been generated and the model has been trained.
After that you get to the satisfying part of scribbeling your own numbers using the Paint SOP. I don’t know how many hours I’ve already spent just drawing digits and random squiggly lines to see how well it works.

Small sample of one of my scribbling sessions. That model hated my 9s.. 😛