Linux and Python for VFX Artists?

You're a visual effects artist, not a programmer right?  Is it worthwhile for a visual effects artist to learn Linux and Python? In my opinion the answer is yes. While you shouldn’t allow learning them to become a distraction to what should be your primary goals of artistic development and mastery of your main DCC apps, a level of competence with Python and programming concepts will only help you in your career development. Depending on your specialty, it my very well also make you a more effective VFX artist. For example, both riggers and FX artists can both benefit greatly by understanding computer science and programming concepts.

While most modern DCC tools don't require that you know programming in order to use them effectively it is still highly beneficial to know programming concepts if one desired to work at a high level in this industry. Python is the language most widely used in the VFX and animation industry for developing custom tools and scripts to streamline production pipelines so it’s the best language to start with. It’s the scripting language of choice for software like Nuke, Maya, Houdini, and more. Learning Python can enable you to create your own tools and possibly more importantly, to automate repetitive tasks. As a dynamically typed, interpreted language,  Python also happens to be fairly easy to learn and use as far as programming languages go.  Python's versatility and simplicity make it a powerful tool for general problem-solving and data manipulation as well. Being proficient in Python can help you troubleshoot issues and optimize workflows in VFX projects. I’m personally of the opinion that simply learning how to think “like a programmer” makes for more clarity of thought when it comes to tackling complex problems. Also consider the fact that the industry can be quite competitive and every advantage that you have over the competition will help you stand out from the crowd. (At some shops, knowing basic programming is even considered table stakes for getting a job!)

Many years ago I was inspired to learn programming and Linux by a couple of mentors. They both gave me little demos, a “kick off” of sorts, which impressed on me the value of learning the technologies. I will forever be grateful to Gene Turnbow and Greg Ercolano for encouraging me to learn how to program and to learn Linux and for taking the time to do those little demos and answer my questions as I started to get into learning. (This was was well before the era of "YouTube” and the web was still very new. Search engines were still pretty terrible back then! That sort of personal mentorship was even more valuable then than it would be today.) Knowing Linux and how to program has been very useful in my professional career and has lead to me getting at, at the very LEAST, my first break in feature animation. (It’s helped me get other jobs since then also.)

While I don’t have the time to be your programming mentor (there are much better programming mentors out there anyway) I can at least encourage you to learn programming just as I was encouraged to learn it so many years ago.

These days basic programming isn't that difficult to learn. Due to the explosion of Internet, web and now Machine Learning development there are numerous online resources for learning to program, most of which are free. (Python is very common in web development and the tool of choice for interfacing with Machine Learning frameworks.)

Once you know the basics of a language and programming concepts you can then leverage some of the Large Language Models like ChatGPT to help you write simple scripts and programs. You are still the boss so you’ll still need to learn the basics of the language to properly USE the code they produce and also correct the inevitable errors LLMs still produce. However with a co-pilot like chat GPT it's amazing how much you can do. The better you are with the understanding the language the better results you will get. (LLMs still can’t write complex programs without the input of a programmer. You can think of them more like a code typing assistant or cheat sheet for a skilled programmer.)

But what about Linux? Most of the large visual effects studios use Linux as their operating system of choice so knowing Linux can be a differentiator when trying to get a job at one of those studios. Unix and Linux have been around for decades so there's tons of great resources available on the Internet that you can use to learn it. Linux is open source as is FreeBSD (Unix) and both have extremely vibrant and generous communities supporting them.

As a final note Linux and Python are used for pretty much all the machine learning projects in existence and a HUGE percentage of Internet and web servers. (Many use programming languages / frameworks other than Python, but nearly all of them run on Linux or Linux containers of some type). The concepts one learns in Linux are transferable to other Unix systems like FreeBSD, which is another perk. My point is, there are many OTHER useful things one can do with knowledge of Linux and programming which might even open up opportunities for you outside of VFX and computer graphics also.

Let’s look at a simple example of navigating in the Linux shell to help demystify it.

While Linux has graphical user interfaces available like Windows and Mac OS, one of its distinguishing features is the availability of an extremely high quality command line interface. (Technically Mac OSX has the same command line interface available also, as it is built on top of a Unix foundation). Many Linux distributions use Bash as their default shell. OSX used to use Bash by default also. It now uses zsh, which is similar enough to not matter for our examples below.  The shell is very convenient for developing and launching simple scripts. The scripts can be written in the native shell language or in a language like Python.

Navigating via the shell uses  the very same shell commands that one might use to construct a script. But rather than creating a single large script the shell runs them one line at a time in an interactive mode. Just like other operating systems you might be familiar with, Linux uses a hierarchical structure of directories for storing files. (“directories” in Linux are what are referred to as “folders” in Windows and Mac OSX). In the command shell, the shell points to at a single directory implicit path which is called the Working Directory. You can thing of the Working Directory as “where I am right now in the file system”. A very common command in Linux is changing this current working directory to another one so that it may become the current directory.The command one uses to change the current working directory is the “cd” command. You use it like like this…

cd /home

The above command sets the current working directory to the directory /home. If you would like to see the contents of the directory you can use the list command “ls”.

ls
aaron/

In this case there was only one file in the home directory and that was my home directory aaron. It’s complete path would be /home/aaron but since the current Working Directory is /home, ls only lists the contents of that dir.

By this point you're probably getting the gist of it. You type a simple command and the shell does something for you. Let's look at a less trivial example. Let's say that I would like the computer to find every PNG file in a particular directory and convert it into a JPEG for me. The computer wouldn't care if there were 10 files or 10,000 files; it would just keep working its way through the files it found until they’d all been processed. Sure… I COULD use programs like AfterEffects or Nuke for the same task but then I’d have to load all the files into the program manually and then save them out. Or if I was using Photoshop I could create an action that would load every file and then save it out. But why not just type in a simple one line command and have the Linux shell do it all for me? Here's an example of what that command might look like. We’ll use the open source OpenImageIO Tool https://openimageio.org  image conversion swiss army knife for this job.  oiiotool is great for VFX work since it knows all about Open Color IO and the image formats we use like EXR and .tx files, not to mention more pedestrian formats like PNG, JPEG and TIFF. In many ways its handling of color is more appropriate for our use case than Photoshop since it uses the same color system that Nuke uses (OCIO).

Here is an example command

find /your/directory -type f -name "*.png" -exec oiiotool {} -o {}.jpg \;

( Replace /your/directory with the directory where your PNG files are actually located. )

It might look a bit intimidating so let’s look at it one piece at a time. Here's what this command does:

• find /your/directory : Used the “find” command search for files in the path “/your/directory”

• -type f : Filters the search to only include files (not directories).

• -name "*.png" : Matches files with the ".png" extension but any base name.

• -exec oiiotool {} -o {}.jpg \; : For each PNG file found, it executes the oiiotool to convert it to JPG. The {} is a placeholder for the file name, and {}.jpg generates the output file with a ".jpg" extension.

After running this command, you should have JPG versions of all the PNG files in the specified directory. They will have the suffix .png.jpg however. We can improve the script with a little more work and only have a .jpg at the end of the output files though. We can even still fit it all on one line!

find /your/directory -type f -name "*.png" -exec sh -c 'file="{}"; base="${file%.*}"; oiiotool "$file" -o "$base.jpg"' \;

The extra functionality we added will strip off the .png before feeding the output name to oiictool. While both these scripts might look a little bit daunting to a novice, they're really not that complex once you understand the constituent parts. Because converting files to different formats and renaming files is a common task, you could take a script like this and put it in a cheat sheet or even save it to a file and use it as a script anytime you needed it. There's not always the need to remember stuff like this from memory. You can look it up in documentation or as I mentioned earlier ask ChatGPT to help you work up a script. I've written similar scripts to the one above probably a hundred times over the course of my career and even I can't always remember it! I still need to resort to Google or in this case I asked ChatGPT to generate it for me. I'll tell you it looks pretty much like the versions I've written from scratch all those times in the past! It's much easier to read code to check it than it is to write it from scratch,  especially once you know the basics of a language. There are also plenty of pre-existing scripts for things like renaming files you can use also.

I encourage everyone to continue exploring Linux, shell scripting and python scripting to at least reach a level of comfort with it.  Knowledge of Linux and programming have helped me make the cut when it came to gaining employment with more than one employer.  Maybe they'll help you stand out also.

Here are some links to some resources to help you get started.

Think Python is an excellent free book for learning both fundamental programming concepts and Python at the same time. https://greenteapress.com/wp/think-python

The Linux Command Line is an amazingly comprehensive free book for learning the Linux command line. https://linuxcommand.org/tlcl.php

Ok, so where can you actually try out the scripts and commands shown in the books? There are numerous free interactive shells available all over the Internet where you can experiment without even installing anything on your computer. 

Python https://www.w3schools.com/python/python_compiler.asp 

Linux https://www.tutorialspoint.com/linux_terminal_online.php

OR, you can install Linux on your machine via a Virtual Machine or Windows Subsystem for Linux (WSL) which is available on Windows 10 or later. https://learn.microsoft.com/en-us/windows/wsl/install 

You can install Python on Windows or in the Linux container you create using WSL. A full tutorial on how to install those environments is beyond the scope of a newsletter but I trust you can figure it out on your own using the above resources if it interests you. I simply wanted to point you in the right direction.

Poll Results

“CopyCat and ML Tools in Nuke” won the poll in the last post’s poll. It was a narrow margin over the second place option but way more than all the others. The people have spoken and ML seems to be something many are interested in. Which got me wondering. Would you like to see me cover ML tools like outside of Nuke? I was thinking Stable Diffusion (and one or more of the various front ends available for it) since it’s Free Open Source and can be installed locally if you have a newer GPU. (There are also portals where you can use it for free online) Please answer the poll below to let me know what you think!