After the Blender Conference, the Geometry Nodes team came together once again to discuss many design topics. This time our focus main focus was to improve support for physics, especially hair dynamics in Geometry Nodes. A few other topics were discussed as well though. You can also read the raw notes we took during the meetings.
The following people participated in the workshop (from left to right): Lukas Tönne, Hans Goudey, Simon Thommes (afternoons) and Jacques Lucke. Additionally, Dalai Felinto helped kickoff the workshop and Falk David joined in every now and then.
Previously in Geometry NodesOur last workshop was 5 months ago. This section provides a quick update on the topics we discussed there. Omitted topics don’t have any news.
Approaching PhysicsAs usual, there are many different perspectives that we have to take into account for when designing how we want to approach physics in Geometry Nodes:
We started out by clarifying that there is a fairly fundamental difference in how to think when chaining multiple geometry operations vs. setting up a physics simulation. The difference is that when creating a simulation, one thinks about the desired behavior (forces, emitters, colliders, …) first, and not so much about the order in which the geometry is actually processed. In fact, the majority of users should only have to care about the behavior without worrying about specific geometry operations.
We therefore want to provide better ways to separate describing the desired behavior from actually implementing the behavior. We call this the declarative approach. It gives users high level control over a potentially very complex evaluation system that makes all the desired behaviors work. The declarative approach can also be very useful for things beyond physics like building a brush engine or scattering system.
To achieve this separation, we will introduce two new socket types: bundles and closures, which are explained in more detail below (exact names are not set in stone yet).
BundlesA bundle is a container that allows packing multiple values into a a single socket. Values of different types can be put into a single bundle. A work-in-progress patch is available already.
Bundles are quite useful to reduce the number of necessary links. For now, we are mostly interested in how they can be used to create a uniform interface for various kinds of simulation behaviors. Each behavior will be a bundle that contains the necessary information for the solver to understand what to do with it.
ClosuresClosures sockets allow passing around arbitrary functions including entire node groups. For example, this allows passing a node group as an input into another group which will then evaluate it. This is an entirely new paradigm in Blender’s node systems, and without being already familiar with the concept of passing functions around as data, it’s not trivial to understand. However, it’s incredibly powerful and allows building much more flexible and user-friendly high level node groups.
In programming, the term “closure” refers to functions which may be passed around as data and can capture variables from where they are created. We have not found a good alternative name yet.
To create closures, we use a new closure zone. It’s a bit like creating a small local node group that can then be passed around. Just using existing node groups does not work, because we need the ability to pass data from the outside into the closure (like in all other zone types). Also, it’s good to have the ability to build the entire node tree in a single group to see everything at once.
Position Based DynamicsThe declarative approach with bundles and closures is generally useful for all kinds of physics simulations. While we want to enable users to build their own solvers, we also want to integrate hair simulation specifically into Geometry Nodes directly.
The hair simulation is designed around a Position-Based Dynamics (PBD/XPBD) solver. This solver has been applied to soft-body simulation, cloth, hair, granular materials and more.
The PBD method is often used for real-time game physics and is relatively easy to implement. It has advantages in terms of speed and accuracy over the linearized velocity-based cloth solver currently used for hair dynamics. There are lots of learning resources and scientific papers on the topic for people to learn more. When first looking into this, we found this overview and this video tutorial series particularly useful.
We will try and implement as much of this as possible using generic geometry nodes. Some parts like collision detection and constraint grouping may require new built-in nodes for performance reasons. This will be decided when we get there.
ListsFor this project we’ll likely need lists in different places, for example to manage a list of behaviors passed into the solver and to process contact points after collision detection. Lists have been a talking point in previous workshops and we don’t have much new information that has not been said before. We went over the set of nodes we’d need, but there were no real surprises there.
Lists are also particularly important for hair, because we need to map generated hair to potentially multiple guide hair strands. Currently, there is no good way to store that mapping which makes any workflow that uses guides, especially for simulation, quite unreliable.
The main blocker to get lists into Geometry Nodes is still the socket shapes discussion.
Socket ShapesThe last blog post contains an explanation of the topic. Last time, we didn’t come to a conclusion for how to deal with socket shapes when we get more types like fields, lists, grids and images. The tricky thing is that we can’t show all information we’d like to with just socket shapes, so we have to decide what we don’t want to show anymore.
Some design work has been done on the topic in the last couple of months and a simple prototype has been built too. We’re now at a point where we are all at least okay with the solution’s tradeoffs so that we can hopefully progress on the topic. Once that is resolved, volume grids and lists are much easier to get into a releasable state.
For Each Geometry ZoneBlender 4.3 comes with the For Each Element zone. While that’s very useful already, there are other kinds of for-each zones that can be useful. One of those is a “For Each Geometry” zone, that we used to call “For Each Unique Instance” in previous workshops.
Its purpose is to iterate over each “real” geometry in a geometry set that may contain an instance hierarchy. Many built-in nodes do this internally already. For example, the Subdivision Surface node applies its effect to all meshes in the input, including those in instances. For various reasons, not all built-in nodes can or should do this. A new For Each Geometry zone would allow adding the same functionality to all built-in nodes and custom node groups which is impossible currently.
This is quite different from the “Instances” mode in the For Each Element zone. If the geometry to be processed contains many instances of the same mesh, the existing zone would run for each mesh separately, while this new zone would only run once, because there is only a single mesh.
There is already some previous design work available in #123021.
Modal Node ToolsWe reconfirmed the overall design for modal node tools from a year ago. Since then, we also noticed that there are two kinds of modal operators in Blender currently:
Both kinds of operators could be created with nodes. However, when we talked about modal node tools so far, we were mainly concerned with the second type. Many use cases of the first kind can probably be solved with gizmos or a gizmo like system. That’s because the interactive part of these operators is mostly just used to control some input values for a non-modal operator.
We also noticed that there are problems caused by fact that all node tools are just a single operator in the end (geometry.execute_node_group), but none of these seem impossible to solve. For example, we want modal node tools to come with their own keymap, but users should be able to override this keymap like any other keymap in Blender. Typically, there is a mapping from modal operator to keymap, but that does not work well here yet for the mentioned reason. Alternatively, it may be a nice solution to attach keymaps to specific assets in the user preferences instead of just to operators.
It can also be possible to register a separate operator for each node tool, but that comes with its own problems. For example, that would introduce yet another way to reference specific asset data-blocks by their operator name and can easily cause operator name conflicts too.
Field Context ZoneWe started discussing a new “Field Context Zone”. The overall design is very incomplete and we don’t have concrete answers to many questions surrounding it yet. The general idea is to give access to the field evaluation context more explicitly.
For example, for a field that’s evaluated on a geometry, the new zone would have the context geometry as input, and would output a field that depends on that geometry. This opens up new opportunities for building fields that would be much more annoying to build before.
The zone would also reduce redundancy in the design of nodes. We have pairs of nodes like Sample Index and Evaluate at Index which are the same except that one has geometry as an input and the other does not. A goal of the zone is that the Evaluate at Index node could be built out of the Sample Index node.
Extra Modifier Evaluation OutputsA limitation of geometry nodes is that it can only output a geometry that is then passed to the next modifier. Sometimes it would be very useful to output other data like another geometry or single values. Those values could become part of the “evaluated state” of an object so that it can be referenced by other objects using nodes or drivers.
This would allow outputting a bunch of vectors from Geometry Nodes which are then used to drive an armature. Additionally, we could allow outputting a bundle of values that is then passed into the next modifier. This way it becomes possible to build more rich modifier stacks without the limitation of having to encode all information in the geometry passed between modifiers.
We could even allow outputting fields and closures from objects (probably with some limitations due to the lifetime of some data). This would allow building all kinds of effector objects that encode some behavior that can be understood by other Geometry Nodes setups. This can also be thought of as a generalization of the existing force field object type.
Internal Data SocketsIn some cases, we want to add functionality that requires passing around data of that we don’t want to expose fully. A good example would be KD trees and BVH trees which allow speeding up algorithms that require finding nearest points or doing ray casts. These data-structures have well defined APIs that we could expose, but exposing their implementation details could make future optimization much more difficult, because optimizations could require breaking files.
It does not seem benefitial to add a new socket type for each kind of internal data. So far we think that it is good enough to only add a single type (with a single color) that is used to pass around all kinds of internal data.
Another use-case that came up in the past is a “Bake Reference” socket that passes data from a Bake to an Import Bake node (once we have that). The tricky thing with an Import Bake node is that it has to be able to read bakes from disk as well as packed bakes. So just giving it a file path input does not work. Reading from files should still be possible of course, but we also need a solution for packed bakes.
Group Input DefaultsEvery input of a node group has a default value. For some types, the default is currently hardcoded (e.g. an empty geometry). Others can be choosen manually in the sidebar where some socket types support more complex inputs. For example, vector sockets can be the position field by default. However, the set of possible defaults is currently hardcoded. The goal of this topic is to generalize the system for defaults to remove limitations.
The overall idea is to have a new “Group Defaults” node that has an input socket for every input of the node group. The default of any input is specified by just connecting the value to the node like in the mockup below.
We could also make it possible for some default values to depend other input values, but it’s not clear yet how much complexity this adds, so that may only be done later.
A tricky aspect is that adding a default to a socket that did not have one yet may override its value in all group nodes that use this group. That’s kind of the inverse of a problem we have already: changing group input defaults are not propagated to group nodes at all. The problem is that we don’t really know if a value has already been modified or not, which becomes even trickier when the node group is linked from another file.
Context InputsThe goal of this topic is to solve the following problems:
What all these issues have in common is that we want to pass information into nested node groups without having to set up all the intermediate links which would cause a lot of annoying boilerplate. Nevertheless, we want to be able to override all these inputs at any intermediate level.
The proposed solution is to generalize the concept of “Context Inputs”. There are many existing context input nodes (like “Scene Time” and “Mouse Position”) already. We also want to add a “Context Input” node for custom inputs. Whenever a context node is used in a (nested) node group, that will automatically create an input for the node group. Group nodes at a higher level can then decide to either pass in a specific value for that input or to not connect it. If it’s not connected, the context input will be propagated further up.
If the context value has not been provided by any node, it’s propagated up to the Geometry Nodes modifier where again users can choose to specify it. If not, we could support reading the value from a custom property of the object or scene.
There is a work-in-progress pull request for this feature.
Modifier InputsWe want to add more features to group inputs in the modifier:
Putting all these choices in the modifier and having them always visible is problematic from a UI perspective. Even now, the button to switch between single value and attribute input adds clutter that is not needed in many cases.
We explored options for how this could work like putting the options in the right click menu, in the manage panel or having “edit” button in the modifier that allows temporarily showing all additional settings. For now, the approach with the right click menu seems best even if it is a little less discoverable at first.
Bundles for Dynamic Socket CountsWhen we explored bundles further, we noticed that they may also provide a good solution for another long standing limitation which we discussed back in 2022: dynamic socket counts. Since then, quite some effort went into improved support for dynamic socket counts and nowadays we have them in multiple built-in nodes like all the zones, “Capture Attribute” and “Bake”. What’s missing is support for building node groups that have a dynamic number of inputs and outputs.
We could allow tagging a bundle input of a node group as an extensible socket. Then from the outside, one could pass multiple values which will become a bundle inside the group. When outputting that bundle from the group, all the values are separated again.
Inside of the group, the nodes would have to process all elements in the bundle. Built-in nodes could do that automatically. For example, when the Capture Attribute node has a bundle input, it could recursively capture each contained field and replace it with the captured anonymous attribute field. Something similar can be done in other nodes that already have a dynamic number of sockets.
Support the Future of BlenderDonate to Blender by joining the Development Fund to support the Blender Foundation’s work on core development, maintenance, and new releases.
♥ Donate to Blender