Creative Work

Muse Spark is a serious contender, for reasons that may surprise you.

AI, Apps, Creative Work, Technology / By
Update: Since this was posted Meta has now announced they are embracing open weights and open source with Muse Spark 1.2 and Muse Glimmer. This greatly shifts the landscape, and in a entirely positive way placing these models in territory which has almost exclusively been dominated by Quen and other Chinese offerings.

There are many factors in choosing a model for development work, as I’ve explored in articles here and elsewhere. Benchmarks are misleading and for most businesses, small teams, and independent developers there is a very difficult balance to be made between costs and capability.

Ultimately, your choices have to be tuned to your own workflow, budget and methods. But there are still some rules of thumb one can follow. One of the big ones is that coding with AI is an iterative process. A single prompt is almost certainly not sufficient to result in a good result unless that prompt is iterated against multiple times behind the scenes. Every iteration costs money. The better the model, the fewer the iterations. But in many cases you are much better off having a lesser model perform more passes than you are having a more powerful model only make a pass or two. AI cloud providers such as OpenAI and Anthropic usually reflect this in their toolsets, offering various schemes to switch over to “less effort” (fewer internal loops) or options for switching to smaller models. The difference in cost between using Fable 5 versus opus 4.8 is significant, and the trend continues as you switch to older opus releases or smaller models such as Sonnet and Haiku.

The problem with this for many is it places the burden of managing costs on the user. Even for those of us happy to tinker with our setups in detail have to admit it’s easy to fall into the trap of paying a premium when you (or your setup) could switch over to a lesser model.

Currently there is a lot of focus on local hosting of open source AIs to get around this trap, and I do this myself. But there are serious tradeoffs inherent in doing so, even with tens of thousands invested in hardware one can’t compete with the sheer power of a frontier model on the cloud… to truly reap the benefits of local AI for any work of weight takes expertise and a lot of effort.

And this is where Muse Spark comes along. Early tests claim scores comparable to Opus 4.8, though users report mid-tier performance in practice. That’s a good starting point for a new line of models.

The differentiation is less about benchmarks and more about price vs performance. Spark is priced roughly 300% less than Opus 4.8 and a whopping 700% less than Fable 5. That’s enough to arguably place Muse Spark #1 when you consider ai-as-a-service coding options.

This has to be an apples to apples comparison still… we are talking about price among the closed-source models, where arguably you are paying to offload much of the complexities to the provider. In this regard, using a service like OpenAI, Gemini, Claude or MetaAI is as much buying into an ecosystem and customer relationship as it is anything else. For many of us, our current providers have won a lot of loyalty Meta has yet to earn. And for many others, the choice to go nearly entirely local is the best solution, despite it’s challenges and up-front costs.

I’m one of the latter, having invested serious money and time into creating my own local coding environment. It’s great, but I come from a big tech background so it’s not something I can suggest for the lighthearted at a scale more than simply setting up a mac with openclaw… things get messy. Fast.

And ultimately, if you are a serious user you’re still going to wind up paying for access to one or more of the Frontier models such as Fable 5, GPT 5.6 Sol, Gemini 3.5 pro etc. They offer deeper reasoning, and this is a hard-to-define factor which affects everything else. A frontier model can comprehend and navigate around cognitive obstacles that lesser models simply can’t iterate thru. So even a power user with a small supercomputer at home can’t avoid the need to at least spend time with the frontier models at key points in planning and decision making.

This is the unavoidable trap of this era of AI: to realize the potential of the technology one must either spend profligately or dedicate time and attention to parceling out tokens where they are spent to best effect. The trifecta of “Good, Fast and Cheap” are still dominant. Pick any two.

Change is good. And change is also terrifying. Keep. Creating.

2D Graphics, 3D Graphics, Creative Work, Film, VFX / By

I’ve been a bit frustrated with this site for a while, because there’s all this fascinating work I do at Meta Reality Labs, that I’d really love to share. While I know many people are reeling under the changes from AI technology as well as the difficult time both game and visual effects creatives have been facing as everything changes beneath us… Please don’t think I’m unsympathetic (or unaffected) when I say this: there has never been a more exciting time to work in CGI.

“Exciting” and “challenging” go together. It means we are in a period of great change. But I’ve found over the years that these are the kind of times when people also tend to do their most meaningful work. This is one of those points when reputations (and fortunes) are made, and when new paradigms and possibilities abound.

If you’re a professional struggling to find work or establish relevance in a world which may not demand some of your formerly most attractive skills, allow me to suggest that you step back a moment and remember that you ARE still relevant. Yes a toddler can now seemingly generate an image or video that would only have been possible through great effort just a few years ago, or vibe code an application that formerly took a team of engineers. All that means is that YOU can do the same, as well as so much more, by leveraging your less obvious skills which have now become massively relevant.

– Eye for quality and detail. – Understanding of process principles to achieve top results. – Industry understanding of tools, methods, ways to approach challenges and visualize your end goals early. – Discipline to slog through difficulties to get to somewhere extraordinary.

You may not be able to do the work you’ve spent decades doing. But that doesn’t mean you lack value. Do NEW stuff. Ride this wave, and go further, faster than you ever dreamt was possible. Or, to be a bit more militaristic in tone: Adapt. Overcome.

Change is struggle. But it’s also opportunity. Figure out where your brass ring is, and reach for it.

Update: It’s all about AI these Days

2D Graphics, 3D Graphics, AI, Creative Work / By

Ok, it’s been some time since I last posted. That is due in part to the fact that much of the work I’m doing for Meta Reality Labs is NDA’d and even the work I do in my personal time often touches on what I’m focusing on at work. I wish I could show off some of the stuff I’ve gotten into recently, it’s pretty amazing.

Here are a few thoughts. The world which has emerged is one of increasing rate of change. For Game Devs, VFX professionals and Technical Artists this change is almost overwhelming. AI is capable of generating astounding visuals in seconds from a simple prompt, as well as models, code, and even entire games. I see this generating a lot of angst 

and fear, and it certainly has changed the recruiting and hiring side of things.

But this kind of change has happened before. The emergence of CGI transformed filmmaking. Computer networks transformed all of

 society. The emergence of digital tools for artists decimated the once huge numbers of “draftsmen” working in architecture, construction and a host of other fields. In recent decades we have seen a LOT of change. Video killed the radio star, and then video on demand killed television, at least to the extent of replacing those media from their seemingly (at the time) unassailable positions as king of the hill.

Today influencers vie with A-listers for viewership, movie box office releases and television series compete for consumer attention with on-demand series, game titles, and social media.

Change is scary and disruptive.

But it can also be astoundingly beneficial. Artists of all kinds shouldn’t fear AI directly. Sure we will have to face uncertainty as job markets change and seek different skills. And that can be incredibly frustrating and challenging. But if you are a skilled professional you absolutely have the insights and talents necessary to thrive in a new world filled with AI. The ability to discern visual quality is still essential. The ability to organize and guide an effort to success is still relevant. “Production sense”, “vision”, an “artist’s eye”, and “creativity” all have as much value as they ever did.

Creative tools have changed and enabled the most unskilled an astounding degree of capability to produce impressive creative works. But that’s good, and doesn’t change the fact that those of us who have spent entire careers focusing on creating such imagery have gained a myriad of skills needed more than ever in assessing and learning to fully exploit these astounding new technologies and tools.

The key to relevancy today as a technical artist or professional creative in general is to ride this wave at it’s cutting edge. Our role in times like this is to identify the new creative opportunities inherent in these changes. Explore new mediums. Guide the engineers towards enabling better quality content as opposed to releasing a societal spam storm of mediocre content. Identify the new bar of quality and use AI and new methods such as gaussian splatting, realtime engines, spatial computing, code and asset generation, image and video generation in new ways. Creative ways.

We aren’t facing a disaster for creatives. It’s just change, filled with disruption of plans to be sure, but also filled with opportunity to be the ones who create entire new media, new forms of artistic expression, and creative content better than anything we dreamed possible in our lifetimes.

Surf this wave of change. Have fun with it. The core of what makes us creatives – vision, insight, keen minds and practical experience – is not at all devalued. Our use cases have just been reset, the canvas wiped clean. Now is the time to make your masterpiece.

Empowering Creativity: Emerging AI Tools for Game Developers and Technical Artists

AI, Creative Work, Tech Art / By

When it comes to game development, technical artists are always on the lookout for tools that can make their lives easier and their work more efficient.

Enter AI. 

From automating repetitive tasks to enhancing asset creation, AI tools can help streamline workflows and free up time for creativity. Technical artists can already use AI tools in a number of helpful ways:

  • Asset generation, including 3d models, animations, rigs, textures, audio and skydomes.

  • Creative inspiration and conceptualization – this can go beyond simple image generation and get more specific with tools like comfyUI, to allow for the creation of specific art styles and applications such as character model sheets, sprite atlases, equirectangular skydomes, and so on.

  • Code exploration and generation. It’s hardly a secret at this point that software development has changed dramatically with LLM tools like Copilot, Windsurf, Codeium, Bolt and Mutable.

  • AI customization and automation is getting very interesting, with resources like huggingface, tensorflow, zapier and more.

  • AI is also a game changer when it comes to communicating ideas and collaborating. I have had some great experiences with Descript and Ryter being particularly notable.

  • And last but perhaps most importantly, AI is a game changer when it comes to research and learning. I regularly use Meta.ai to explore technical topics and other go-to tools are Consensus and Perplexity. If you are using google to try to learn about something or research a topic, you are missing out on the power of AI in helping you obtain the information you need to access and understand.

The number and nature of the AI tools available are rapidly changing. They’re not just about saving time; they’re also about pushing the boundaries of what’s possible in technical art. Currently we are in a period where we are confronted with many different tools but have yet to bring them together into cohesive workflows. As such, I’m just going to touch base on a few areas I’m keeping an eye on. Think of this article as a starting point, I’ll add more to this later and hopefully end up with some tangible assets and workflows worth considering.

Infographic on AI tools and tech art

Unity ML-Agents

Unity ML-Agents are a powerful toolkit that allow developers to create intelligent agents within their games. By leveraging machine learning techniques, developers can simulate complex behaviors and decision-making processes.

  • Intelligent Agent Creation: Unity ML-Agents enables developers to train agents using reinforcement learning, allowing them to learn from their environment and improve their performance over time. This can lead to more realistic and responsive non-player characters (NPCs).

  • Machine Learning Applications: Examples include creating NPCs that adapt to player behavior, developing dynamic difficulty adjustments, and simulating realistic character interactions.

  • Gameplay Experience Enhancement: By integrating intelligent agents, developers can create more immersive and engaging gameplay experiences that respond to players in real-time, ultimately leading to higher player satisfaction. For more information, visit the Unity ML-Agents documentation.

 

NVIDIA GameWorks

NVIDIA GameWorks provides a suite of tools and technologies designed to enhance the graphical fidelity and performance of games.

  • Key Features: GameWorks offers tools such as PhysX for realistic physics simulations, NVIDIA HairWorks for lifelike hair rendering, and NVIDIA Flow for fluid simulations.

  • AI-Driven Solutions: One of the standout features is Deep Learning Super Sampling (DLSS), which utilizes AI to upscale lower-resolution images in real-time, providing stunning visuals without compromising performance. For more details, check out NVIDIA GameWorks.

  • Successful Implementations: Popular titles like “Cyberpunk 2077” and “Control” have successfully implemented NVIDIA GameWorks, showcasing the dramatic improvements in graphics and performance that these tools can achieve.

 

Houdini

Houdini is a 3D animation and visual effects software known for its procedural capabilities and integration of AI features, making it invaluable for technical artists.

  • AI-Integrated Features: Houdini offers tools that leverage AI for tasks such as automating texture generation, enhancing simulations, and optimizing workflows. This can greatly reduce the time artists spend on repetitive tasks. Of particular note are the

  • Procedural Generation and Simulation: The procedural nature of Houdini allows artists to create complex environments and effects efficiently. This streamlining can significantly reduce the time spent on manual adjustments.

Of particular note, the Houdini node collection known as MLOPS are hugely useful to technical artists. Houdini version 20 also added their first official machine learning node, the ONYX inference SOP.

 

For further details, visit the Houdini documentation.

 

Meta Developer AI Tools

Meta developer AI tools, such as Llama, represent a significant advancement in how developers create and manage their workflows. These tools utilize large language models (LLMs) to automate and enhance various aspects of game development. Meta has embraced an open source and collaborative stance with developers which is worth watching closely as fertile ground for useful tools, SDKs and APIs.

  • Llama Overview: Developed by Meta, Llama is a family of state-of-the-art open-source LLMs that can assist developers in a myriad of tasks, from coding to documentation.

  • Applications in Game Development:

    • Code Generation: Llama can generate code snippets or entire functions based on natural language prompts, facilitating rapid prototyping and reducing the time spent on boilerplate code.

    • Debugging Assistance: Developers can input error messages or code snippets, and Llama can suggest potential fixes or improvements, streamlining the debugging process.

    • Documentation Creation: By summarizing code and generating comments, Llama aids in creating and maintaining documentation, which is crucial for team collaboration.

  • Integration: Llama can be integrated into various IDEs and development environments, enhancing the overall workflow without disrupting existing processes. For example, integrating Llama with Unity or Unreal Engine can significantly improve productivity by allowing developers to focus on creative tasks rather than repetitive coding.

For more details, you can explore the Meta Llama documentation.

 

Additional AI Tools

In addition to the previously discussed tools, several other AI solutions can enhance the game development process:

  • Windsurf: This tool assists in optimizing and automating the development of game assets, improving workflow efficiency. Learn more.

  • ComfyUI: An intuitive interface for managing UI elements in games, allowing for rapid prototyping and adjustments.

  • Rytr: This AI writing assistant can help developers and writers generate engaging narratives and dialogue for their games. Visit Rytr.

  • Descript: A powerful tool for audio editing that utilizes AI to streamline the process of creating and editing voiceovers and sound effects. Explore Descript.

  • Zapier: Automation software that can connect various applications and tools used in game development, helping to manage workflows and tasks more efficiently. Find out more.

 

 

Working in VR

Creative Work, Virtual and mixed reality / By

It’s been a while since my last post, mainly because I’ve been busy at Meta. Working in VR on a daily basis has been fascinating. Meetings in VR, working in a virtual office with a Quest Pro, and (now that the pro can be seen in public) traveling with that same virtual office.

There are a lot of things I can’t share but one thing which strikes me is the sheer speed at which VR is becoming a transformative tool for all kinds of business. Interestingly enough, the “standard” business use cases are emerging more rapidly than many of the creative scenarios.

In hindsight this makes sense… You can get a lot of work done with 2d applications on laptops and mobile devices already. While often creative work requires high-end physical setups with resolutions and color clarity not yet available in VR.

What VR is bringing to the table is that you can now use your ”standard” workplace applications tools in virtual spaces… which you can take with you.

By this I mean your workspace is not limited to your portable screen any more. You can take along your whole office (albeit a virtual one), with arrays of huge monitors around you to multitask as you prefer. And already you have increasing control of what the rest of the office around you looks like… a fireplace nearby, or a night sky and aurora above you… a music system, knickknacks on a shelf, photos of a loved one, a working clock on your desk, and so on.

The point is your virtual space can exist wherever you do. In a hotel room, airport terminal, or the cramped confines of airline seating (where the ability to be anywhere other than 9 inches away from the seat in front of you becomes a godsend..) Tired of the current layout or your view? You can change it around whenever you choose. Need an additional monitor? Just spawn another one.

With mixed reality, you can increasingly blend the real world around you with the virtual. You can see your real hands typing on a virtual keyboard. You can remain in VR but confidently move between your couch and your desk. Don’t worry, if your pet walks into your space you can see it, too.

And you can collaborate with your coworkers in your virtual office space, regardless of where you are. I do the majority of my work from my home and meet in VR a lot. I tend to prefer it to zoom meetings.

Sure, you are in a somewhat cartoony place with somewhat cartoony coworkers and self. Surprisingly that really doesn’t matter – you get used to it and even better you don’t have bad hair days or need to worry about what your physical surroundings may look like.

The sense of presence makes a big difference. The nature of discussions is impacted I think, and in a positive way. For better or worse, you are in the meeting. And I find the additional engagement this sense of presence delivers causes me to participate and remember the meeting more.

This sense of additional engagement from a VR presence has been observed in a number of studies which have measured considerable increase in participant’s retention of information in VR versus videoconferencing. And unlike physical real-world meetings participants can be scattered across the planet.

The first time I went to a real-world meeting with my team at meta I was immediately comfortable and productive with my teammates – despite never having actually been in the same room with them before. I knew them already, from VR. Their body language was familiar, their whole presence was known to me already, the fact that we were physically in the same room for the first time was unremarkable.

That’s not to say face-to-face communications are going by the wayside, not at all. Businesses are discovering that it’s critical to make sure closely partnered coworkers get some “real world” time. VR cannot replace face-to-face interactions entirely. But it can make remotely dispersed teams more comfortable and productive.

From any physical location you can all group up. You can read each other’s expressions and body language and feel their proximity. It doesn’t matter if you don’t look your best because your avatar looks fine. Meeting in VR gives you the immediacy of dropping into a videoconference with the familiarity (and retention) of a face to face meeting.

I can tap my headset and see the “real” world around me if I need to. My passthrough image on the Quest Pro is accurate and fully stereoscopic so I can comfortably reach out to grab a cup of hot coffee from a flight attendant, or pet my dog on the head in my home office… before jumping back “into” my VR space.

In a VR workspace I can mute myself but remain in the virtual room. I can see my laptop screen or project it onto the VR whiteboard so everyone can see it. I can draw on the whiteboard from my desk, point at it with a “laser pointer,” or get up in front of it to present to the others. Attendees who call in from “2d” (aka video participants using their cellphones or webcams) are there too.

And like the internet once did, this is going to change everything.

It’s transformative. This is why companies are purchasing headsets designed specifically for this kind of working, and why companies like Microsoft and Accenture are moving to bring their tools and workforces into the VR space.

Like the internet when it was new, you’re going to have to experience it for a while before the benefits of this new kind of media really sinks in. The tools and workflows are going to change and improve. Offices will get rid of the miles of cubicle spaces that exist today and instead emphasize conference spaces and comfortable temp locations for a more mobile and dispersed workforce.

Home workspaces will become more important, while costs once applied towards physical office infrastructure will be reduced to a degree and applied towards equipping a more remote workforce, and supporting periodic meetups.

This isn’t going away: you are going to experience it (if you haven’t already.) And in time literally everyone will. It’s that transformative.

Like the early internet, or smart phones, DVDs, streaming, the web and online shopping this inevitable, and it’s coming soon.

The first years are going to be marked by frustrations and constant change, but that’s part of the fun. Fortunes will be made and lost, new markets will emerge, and surprises are going to happen every step of the way… Enjoy the ride!

Procedural Glyphs, another method

2D Graphics, Creative Work, Houdini / By

I thought I was done with the procedural glyphs project but it occured to me there was another approach I could take and I had to give it a try. The results are a bit closer to feeling handwritten. What would be really interesting would be to develop a continuous cursive-like script with some form of punctuation.

Update: Cursive writing GET.

Misc Shots

3D Graphics, Creative Work, Examples, Film, VFX / By

I haven’t edited an “official” demo reel in a very long time. People know who I am, and hire me for projects based on peer testimonial and having seen past work directly. A reel just hasn’t been needed, since I’m fortunate enough to be a freelancer who gets approached regularly. So when I started teaching students who knew nothing about me I had the challenge of presenting myself to an entirely new audience who were eager to see some work I’ve done in films. I really haven’t had time to edit much, so instead I spent an evening grabbing various shots from films and the like. Stuff I’ve either worked on directly, supervised, or had some hand in.

I have only grabbed a portion of the films I’ve worked on, plus some TV and game stuff, pretty much in the order they were stacked on my desk. The choices are fairly miscellaneous – at the moment I just don’t have time to go through film after film and say “oh yeah, I did that,” then tighten it all down into an edit with music and polish. I know, this breaks all the rules where I’m supposed to present only the very very best, nothing less. But the policy that seems to work best for me is to just be open, good or bad just get it out there and get back to making the next thing. I will never be a charismatic showman or clever self-promoter. I’m a creative geek and frankly I like showing failures, too. It’s part of the process, and that’s what really motivates me.

So all of that said, here’s about 6 minutes of footage and, well, stuff. I basically play it in the background when I introduce myself to students, so it is what it is. Enjoy!

Article on 80 Level

Creative Work, Game Effects, Houdini, Miscellaneous, PopcornFX, Unity, VFX / By

https://80.lv/articles/using-hooke-s-law-to-create-vfx-effects/

I wrote an article for the game-and-vfx website 80 Level a while back, and it’s gone live. The topic is Hooke’s law as implemented in Popcorn FX (and Unity’s VFX graph.) It’s brief but has been well received, I appreciate the feedback I’ve gotten. A second article for 80.lv centered around Newton’s universal gravitation as implemented in PopcornFX (and Houdini this time) is in the works.