Vibe Coding in After Effects: We created an AI plugin with NO coding experience!

0
2


It’s now easier than ever to make changes and add features to After Effects with a little help from AI! After a little investigation, we’ve tracked down the best tools to create our own expressions and plugins with AI, with absolutely no coding experience.

Recently we ran an article delving into vibe coding (Using an AI assistant to code a program, website or tool). It’s a hot new term that’s catching on fast, and its not just for website design and gaming. Devs and dabblers at all levels are turning After Effects upside down with AI-assisted scripts and patches, making it into their own hot-wired, kit-bashed frankenstein of everyone’s favourite motion tool.

What we love about vibe coding is that it gets AI working on the tricky right-brained backend of the software, leaving the more creative elements for us creative humans!

We looked into What vibe coding means for motion design and how you can start today. Plus, we spoke to some industry pros about their take on the colliding worlds of AI and AE.

Vibe coding isn’t a magic button—there’s definitely still human effort involved. But with the right AI tools, you can solve tricky problems, pull off clever tricks, and even build tools that might make you some cash. Just be ready for a little learning (and a little patience), especially if you’re new to coding.

Venturing to the new frontier

Okay, back to the matter at hand. After a little investigation, we identified two distinct avenues for vibe coding in After Effects: one simple and the other more advanced. Let’s use Adobe’s terminology and call them Commands and Plugins respectively.

  • Commands means writing expressions. Bits of code we can copy & paste into the properties of a layer like position & rotation.
  • Plugins covers anything with a UI, like extensions, effects and ScriptUI panels.

Starting simple: Commands

Writing commands has become exceedingly accessible. Using cheap and/or free AI assistants like ChatGPT, we were able to chat our way to solving a handful of creative challenges in under half an hour. We managed with minimal coding experience, but it helped to be familiar with After Effects. If you don’t know the boundaries, it can be hard to push them, and knowing AE terminology helps a lot. Here’s what we asked Chat GPT to do (running on a free account):

  • Draw lines between multiple moving layers, and find midpoints (or an exact percent of the distance) between two layers.
  • Create a per-character text animation, but with variable fonts, compensating for letter size changes.
  • Create a radiating Blend between 2 shapes, where the interpolated instances adjust in realtime to position, size and rotation changes to the 2 base shape layers.

How to speak to robots

Generic AI assistants have no concept of how we want them to speak, but as we discovered; a casual, friendly and descriptive approach to our prompts yielded the most helpful results from ChatGPT. We also included suggestions of methods we think should work:

The assistant may not get it right the first time, but if you are clear about all the moving parts required, and clearly describe any issues with the scripts it provides, AI can quickly solve challenges that would otherwise be abandoned as too hard or out of scope.

Chat GPT has an especially good memory of previous interactions, and can repeat similar solutions from previous sessions. This is very helpful when building a relationship with your artificial assistant.Another tool worthy of mention at this point is Klutz GPT from Hyper Brew, which brings Chat GPT right inside After Effects via a dockable panel.

Stepping it up: Plugins

As Motion Designers, We often wish we could create a tool that we can use again and again, share amongst teammates, or even monetize! That’s when plugins come in handy. There are a number of approaches to creating plugins and we found that there are AI assistants waiting to hold your hand wherever you go. Even if you don’t understand the code, AI Agents will tell you exactly what they are doing and the function of all files they create.

Software

OK, so what about the software; what does one actually do vibe coding with? Well it can be any AI assistant, but the one that has everyone reeling right now is Claude*. More specifically Claude when used as the onboard assistant in the development tool Cursor. Cursor is an IDE (Integrated Development Tool). IDE’s are apps that developers code with for myriad software applications. Cursor’s a free desktop app, available online, with paid subscriptions for advanced features. There are many different IDEs which do the same things (i.e. BS code, Visual Code, Windsurf & Tempolabs) but devs are flocking to Cursor for its price point, UX and integration with Claude.

*with Google’s Gemini a significant contender.

How to speak to robots, part 2: Creating a plugin

We decided to give the Claude+Cursor workflow a try ourselves and were amazed at how quickly we were able to create our own effect, running as an extension right inside After Effects. We told Claude in the chat window that we had no coding experience, but we had the Adobe SDKs handy. The chatbot helped us through everything and we only had to retry a few times (human error), with Claude suggesting a number of fixes until it struck upon the a file naming error and our effect was working!

In order to start simple, we began with the following prompt, mentioning a CEP, (bundle of files) because it was something we came across in our brief research: I am trying to create an effect panel for Adobe After Effects. I suspect this will need to take the form of a CEP. I do not know much coding and will need a lot of help.

At this point we did not specify a function for our panel, instead letting Claude dream up an idea. This freed us to focus on walking before we attempted running. The AI Agent came up with a panel that would animate a blur on and off, based on a couple of parameters:

Now it’s just a matter of fine tuning. This is one of those things you can do 90% of in the first day, then the last 10% can take weeks, depending on how much of a perfectionist you are.

Creation of our own plugin: AEShapeFilly

Having successfully created a Plugin of Claude’s invention, it was our turn. We decided to attempt making a plugin that would take any shape object and fill it with circles and squares of varying sizes.Just as an example, we tried making a tool to fill existing shapes with a pattern of circles. Let’s call it “Shape Filly”.We started by creating a new local folder named “AEShapeFilly” and opened it in Cursor.

To open the Claude assistant in Cursor:

  1. Toggle open the AI Pane.
  2. Select Claude-3.5-sonnet from the drop-down menu as your Agent.

We downloaded the After Effects Panel SDK (here) and added it to the folder (you have to convert the pdf to rtf, or it can’t read it), We then asked Claude to read it and populate the folder with the necessary bundle of folders to begin an After Effects Plugin:

It pays to tell Cursor not to do anything more until asked.Next we define the intended Plug-in. Try to be specific and brief. Ask Claude to repeat it back before proceeding and suggest any additions, which you can agree to or veto.

Claude came back to us with a great summary and a few additional features, plus some final confirmation before creating the first version of our new plugin.As per the SDK, it created an index.html file and the correct folder structure with everything in its right place, as well as a readme file.It explains everything it has done within the chat so you don’t really need to look at the code, unless you really know what’s going on under the hood.We confirmed that it sounded great and asked how to install. Claude gave us instructions.

Sometimes it has to ask you to run a command to create some elements. just hit Run Command unless it’s asking to destroy all humans or something.We followed a few prompts from Claude, like adding a CEP directory in our After Effects Extensions folder, and presto, it was time to test our new Plugin!

The UI looked great, but when we hit apply nothing happened. Time to report back to Claude and start troubleshooting.

At this point, Claude usually adds an error log to help sniff out the issue. It also forgets to actually reinstall the updated builds as we go through testing, so be sure to remind it.After a little back and fourth, we got a result—our shape layer was populated with dots, as requested! But there were 2 issues to fix: overlapping dots, and empty areas near the curved borders of the shape:

In the next few iterations, we went back and fourth with the AI, ironing out issues around distribution of the dots until we had something that was looking pretty neat!

The final version of our Extension Panel: Shape Filly, made with AI! Of course, we had to try it out with the Envato logo.

The future of vibe coding is bright

If you’d like to see what vibe coding in the world of building websites looks like, check out our short course with Adi right here:


And this final takeaway from James Whiffin really sums up our investigation into vibe coding for After Effects:

“The barriers are reducing every year, but I think AI tools have sped this up significantly […] AI is just a tool (a very powerful one) but it’s how you use it. I would recommend people use it to get their feet wet in tool development, but to really dig deeper beyond relying on LLMs if they are serious.”

We would say this experience has inspired us to explore this avenue further and find more possibilities of Vibe Coding for After Effects. It has also clarified the certainty that Motion Designers must have a base understanding of code if they are to survive the future we see for the discipline.



Source link