Understanding Elixir (Re)compilation - ElixirConf 2018

Some projects take a long time to recompile, this happens because each time we make a change in our code that change triggers other files to recompile even when there is no direct relationship between these modules.

On this case, this recompilation happens because of macros.

When we use a macro from a module and that original module changes, all the modules that used this macro have to recompile to have the last version of the macro. This happens because macros code are injected to the modules that use them at compile time, so for each change on the original macro we have to recompile both the module where the macro is and the modules that use it.

Let’s assume we have these three modules:

In metaprogramming as we inject code from another module to the present one, each time we update the module with the metaprogramming construct we have to recompile each module that uses that macro defined there.

Now watch the following terminal session:

If you want to know which files just changed while compiling your project run on your project root (before compiling):

inotifywait -rm -e MODIFY ./

or better yet if you want to focus on the .beam files that changed:

inotifywait -rm -e MODIFY _build/* | grep '/ebin/ .*\.beam$' &


Run-time dependencies and Compile-time dependencies

“If module ‘A’ calls a function from module ‘B’, we say that ‘A’ has a run-time dependency on ‘B’.

If module ‘A’ calls a macro from module ‘B’, we say that ‘A’ has a compile-time dependency on ‘B’ ”

As you watched on the terminal session this compile-time dependency means that each time we compile the module with a macro we have to recompile all modules that use this macro after that.

Now to view what module depends on which use:

mix xref graph

As you can see it sends back not only the dependencies but their type! (compile/struct/run-time)

How are Compile-time dependencies created(13:33):

  • A module is {imported, required, used}
  • A struct is referenced with the %Module{} syntax (not since elixir 1.6)
  • When Implementing Protocols
  • When Implementing Behaviours
  • When an atom is **seen** on macro-expansion which can lead to…

Transitive Run-time Dependencies

Let’s say we have Module 1 which depends on the run time on Module 2, And we have a Module 0 which depends on compile time on module 1. Now as the macro in module 1 could have anything (other macros or modules) inside of it, each time we compile module 1 or its dependencies (in this case module 2) we have to recompile all the modules that depend on module 1 (module 0 in this scenario), which creates this transitive dependency (it’s called run-time because the dependency that could break it It’s actually the run-time dependency between module 1 and module 2).

Note: running mix xref graph doesn’t show explicitly transitive run-time dependencies

A general rule is:

“If module ‘A’ is modified, every other module with a path to ‘A’ that contains at least one edge labeled “compile”, will be recompiled”

Given the following three modules:

Which modules do you think will recompile when I change the last module?

Transitive dependencies from libraries can lead to Cycle dependencies. And these are really bad as you only need one compile-time dependency for the whole cycle to become a compile-time dependency, which can make code from outside the cycle to have this compile-time dependency too.

If you want a graphical representation of your dependencies use the commands:

mix xref graph — format dot

and then type the following to get an image:

dot -Tpng xref_graph.dot -o xref_graph.png

For our example we got from our examples:

and I highly recommend you to try:

mix xref graph --format stats

Last resort

We could make the run time dependency go away if we modify the module in the middle like:

“Although this is possible, you need to make sure that it is safe to “break” the dependency. If you call anything on the “concat”‘d module, you risk having “stale” .beam files, which might present very hard to reproduce “bugs”. Use Module.concat/1 only as your last resort.”

There’s also another option but it’s not recommended.

Special Thanks

To Renan Ranelli for giving this talk at ElixirConf 2018, the content and the examples were from the talk which was based on this blog post: http://milhouseonsoftware.com/2016/08/11/understanding-elixir-recompilation/

Leave a Reply

Your email address will not be published. Required fields are marked *