Push Stacks for Better Asset Management + More #183

wrabit · 2024-10-06T08:55:11Z

wrabit
Oct 6, 2024
Maintainer

A feature that I have long felt missing in cotton (and django) is the ability to push content from inside a component to anywhere else in the template. This is comparable to Laravel's 'push' 'stack' concept. To me, this particularly comes into its own when you have a component for example a Tooltip with its own 3rd party script or stylesheet, which may be used many times on one page and across multiple pages the frontend.

Attempts have been made to bring this to Django, for example django sekiszai, but there are limitations, tricky conditions and I think only works on the back of the blocks + extends pattern.

The problem

When we build out our component, in attempt to keep all effects encapsulated in the component, we may do this:

<!-- tooltip.html -->
<div>{{ slot }}</div>

<script src="tooltip.js"></script>
<link href="...." />

As you will know, using this multiple times is pretty inefficient, duplicating assets in the page not to mention possible clashes of js instances.
Due to Django's "top down" rendering pattern, we are unable to affect state of a component already rendered earlier in the process

The current solution

Set a context variable that places certain assets in the template on certain pages
Conditionally render the assets with some logic in the head or foot of the template

Usually, any solution causes us to break component encapsulation and brings in the need to maintain dependencies somewhere else in the project.

push/stack pattern:

<!-- base.html or anywhere else in another template -->
<html>
<head>
  <c-stack name="head" />
</head>
...

<!-- comp1.html -->
<div>Wizz bong</div>

<c-push to="head">
  <script src="lib.js"></script>
  <script>
    myLib.init() // etc
  </script>
</c-push>

Features notes

2 new cotton tags <c-stack name="[stack name]" /> and <c-push to="[stack name]"></c-push>
It will not permit duplicate pushes to the same stack, so you can safely define this in your component and use the component multiple times, it will only display the one instance
Works well also with Alpine.data component definitions, stylesheets, any arbitrary code that needs to be somewhere appear somewhere else in the template
This approach skips the need to maintain any type of 'component registry' to collect and declare assets, as with other solutions.

How it works:

The easiest way to hook into post rendering in Django is middleware, so this feature (can be optional) would require adding a middleware entry.
The processed rendered output is cached, much like the way Django's cached loader works. It also breaks when there is a template change, so it will work in development and production.

Further enhancements

This approach has the opinion that by default we should assume pushes to the same stack are made unique. There may be cases where duplicates are desired, this could be dealt with later with a 'multiple' attribute on the stack or push.
I think this feature further justifies the need to have some type of 'cache warming' command so it can be run on deployments to keep optimal experience page speed.

Concerns

(I do have this as a working branch with tests)

Are there any limitations, cautions we should be thinking about with this approach, particularly I guess around the middleware?
We currently use <c- pattern for components and also a few features like <c-slot, <c-component, <c-vars. This is proposal will also add <c-stack and <c-push. Using more words in the component namespace prevents the developer from using them for component names. Do we consider this ok? I was thinking of an alternative syntax for the library-related terms like <c:slot, <c:vars etc.
The approach of caching rendered content should be ok but there could always be edge cases I am not thinking about at the moment.

Voicing this for anyone interested and specifically want to kindly ask opinion from @mattbha and @copyfactory!

samGbos · 2024-10-06T12:58:10Z

samGbos
Oct 6, 2024

We wrote a home-baked version of this at $WORK, and I can confirm that it's an absolute killer feature I can't live without. +1

0 replies

copyfactory · 2024-10-07T22:36:34Z

copyfactory
Oct 7, 2024

Hey @wrabit ,

Adding a middleware to every request adds some overhead when it may not be needed.

Here are a few different ideas to noodle on.

If the content of <c-stack> is always static (say a stylesheet or JS file) we could add a hook in the app config to 'build' the stacks at runtime and therefore never need to run middleware. This has the unwanted side effect of always having the FULL stack rendered once but provides the DX mentioned in the proposal.
A more radical idea is to move this logic to the client, in HTMX there is the out-of-bounds swap where you can specify an append-only strategy. Or in Alpine there is the x-teleport directive. I'm not suggesting that Cotton adopts one of these frameworks but it could be worth exploring. Another consideration for this is partials. If I understand the implementation correctly, the full stack needs to be rendered everytime? This means that if I just return a template partial of a say a button with the tooltip, how does the stack get appended to?

2 replies

wrabit Oct 10, 2024
Maintainer Author

Thanks for the notes @copyfactory. I considered the idea of some automated step to gather all push/stacks in a template stack. The 'downside' is you won't be able to use any context inside the pushes since it will be before rendering. (If I get your proposal correctly). But I am still wondering if having no context inside a push is a deal breaker. You wouldn't for example do something like:

<c-push to="head"> <script> some_init_script_for_{{ id }}() </script> </c-push>

If I can help it, I want to steer away from having a js dependency or layer in cotton for the basic functionality. I'm not sure I understood your partials idea...

FWIW the overhead is really quite minimal from initial testing #183 (comment)

fbinz Feb 12, 2025

I'm not sure I understood your partials idea...

Maybe just to pick up on the topic of partials (maybe from django-template-partials).
I don't want to put words in @copyfactory 's mouth, but I guess the issue is this:

// base.thml
<html>
  <head>
    <c-stack name="head" />
  </head>
  <body>
    {% partialdef component inline %}
      <c-my-component />
    {% endpartialdef %}
  </body>
</html>

// my_component.html
<button onclick="handleClick">Click me</button>

<c-push to="head">
  <script>
    function handleClick() { console.log("clicked"); }
  </script>
</c-push>

What would happen, if you'd only render the "component" partial, i.e.:

def some_view(request):
    return TemplateReponse(request, "base.html#component")

Where would the "push" go?
To me it seems, like just not-rendering the contents of the <c-push> element would be not surprising.

Still, there might be use-cases for having some kind of fallback stack, if there is no other stack of the same name.
For example, HTMX has the head-support extension, which allows partials to include <head> elements which get merged with the existing <head> element.
In that case, it might be conceivable to have a <c-stack name="head" fallback> component, that is only used, if there is no other stack of the same name.

mattbha · 2024-10-07T22:41:53Z

mattbha
Oct 7, 2024

To be honest I don't think this is something I personally can find a use case for. By design I always avoid inline js and scripts linking.

I prefer to use the tailwindcss+alpine+htmx stack, so the literal only thing I'm writing is python and django template html. (I've tried a large number of different things, and this is by far my favorite. django-cotton is the missing piece in making this all flow beautifully.)

Typically I'll just bring in all my static assets in base.html and serve them there. They are downloaded one time and are cached on all follow up reads -- so even if a page doesn't need them, there is no need to worry about removing it.

<!-- tooltip.html -->
<div>{{ slot }}</div>

<script src="tooltip.js"></script>
<link href="...." />

is something I personally avoid, and just use alpine or htmx to handle via pure html.

2 replies

copyfactory Oct 7, 2024

I am the same, compressor to serve a single JS file (which is tiny since its just Alpine and some plugins) and a single CSS for tailwind.

wrabit Oct 10, 2024
Maintainer Author

I have reached for both approaches and it varies from project to project which one I go for, sometimes a mix because I might be using some re-usable components from another library. Maybe I only need it one page and this would ensure that asset (and others) aren't included unnecessarily on every page.

If it's more of an app-wide dependency then sure, into compressor and keep things minified, combined and cached.

mattbha · 2024-10-07T22:52:55Z

mattbha
Oct 7, 2024

Also to the point

in attempt to keep all effects encapsulated in the component, we may do this:

<!-- tooltip.html -->
<div>{{ slot }}</div>

<script src="tooltip.js"></script>
<link href="...." />

In my mind, encapsulation isnt as clear as it could be as the behavior is decoupled. Which is why id prefer alpinejs (or htmx) here -- locality of behavior is more preserved.

https://htmx.org/essays/locality-of-behaviour/

Just for completeness, here's how I might add a tooltip to a button.

<div x-data="{ tooltip: false }">
    <button @mouseenter="tooltip = true" 
            @mouseleave="tooltip = false">
        Hover Over Me!
    </button>
    <div x-show="tooltip" class="z-50 absolute bg-white border-graphite border-2 rounded p-4 mt-1">
      My tooltip text
    </div>
</div>

And of course with cotton, you could break this into components. The main benefit is the business logic is very clear (my_view.html) and you can trivially place tooltip buttons around everywhere (<c-button>). Or even inline some non-dynamic tooltips. (<c-tooltip>). So it is very scalable, and approachable for a developer. (versus the above) This is just an example.

All the tooltip "behavior" is in one file, button.html.

my_view.html

<c-button tooltip="My tooltip text">
    Hover Over Me!        
</c-button>

button.html

<div x-data="{ ttvis: false }">
    <button @mouseenter="ttvis = true" 
            @mouseleave="ttvis = false">
        {{ slot }}
    </button>
    {% if tooltip %}
        <c-tooltip x-show="ttvis">{{ tooltip }}</c-tooltip>
    {% endif %}
</div>

tooltip.html

<div class="z-50 absolute bg-white border-graphite border-2 rounded p-4 mt-1" {{ attrs }}>
  {{ slot }}
</div>

2 replies

wrabit Oct 10, 2024
Maintainer Author

<div x-data="{ tooltip: false }">
    <button @mouseenter="tooltip = true" 
            @mouseleave="tooltip = false">
        Hover Over Me!
    </button>
    <div x-show="tooltip" class="z-50 absolute bg-white border-graphite border-2 rounded p-4 mt-1">
      My tooltip text
    </div>
</div>

Once your [any component] grows and your x-data becomes larger, you may reach for the typical extraction as either a function or Alpine.data. Then you have naming conflicts of duplicated methods in the document, but you also have a lot of repeated code in the html. Push stacks would keep the content unique, so only one funciton/Alpine.data definition would be in the resulting html!

mattbha Oct 10, 2024

Fundamentally I try to keep components simple and avoid premature abstraction. My typical cotton component is probably 5 lines of html.

The repeated code in the HTML I do not think is a large concern; over-the-wire it's all compressed anyways. I'd take verbose repeated HTML over having to make a change in a project with tons of custom js/css wires any day.

Of course there are special scaling cases (a 100 row table with 20 columns where each cell has 3 tooltips ie), where this would help a lot.

I still think its a valuable feature to have for cotton even if I personally would not use the technique/middleware.

samGbos · 2024-10-07T23:18:53Z

samGbos
Oct 7, 2024

For what it's worth, here is an example of how we use our similar version (which is based on custom template tags):

<!-- datepicker.html -->
<div class='datepicker'>datepicker html here</div>

{% render_at 'css' %}
<style>
  .datepicker { ... }
</style>
{% endrender_at %}

{% render_at 'js' %}
<script src="{% static "js/big_file_only_needed_on_pages_with_datepickers.js" %}" type="module"></script>
{% endrender_at %}

And then somewhere in a base template:

{% render_target 'css' %}
{% render_target 'js' %}

A couple nice things about this:

The styles that apply to the component are in the same file as the html for the component, which makes it easier to know that the CSS in that file is just intended for that component. This can still be paired with a global set of utility classes like tailwind, but often it's nice not to entirely rely on utility CSS to get everything done.
big_file_only_needed_on_pages_with_datepickers.js is only loaded on pages that need it, and it is automatically included on every page that includes datepicker.html. This is especially nice when including a widget from a form defined in python, and knowing that you don't need to remember to also add a script include to every page it will be used on (and the reverse - forgetting to remove the script include when you are no longer using that widget)

In terms of performance overhead, our naive implementation is based on middleware and takes ~dozens microseconds to run, which is likely fast enough for most people using Django.

1 reply

wrabit Oct 10, 2024
Maintainer Author

You list the exact benefits to this pattern for me

wrabit · 2024-10-10T17:09:25Z

wrabit
Oct 10, 2024
Maintainer Author

I have times where I reach for both of these patterns. In one app we use compressor for the main assets, we do have a couple of Alpine.data()'s outside of that, but for the requirements of this app it's fine. In this case I would reach for push stacks to keep the definition close to the component.

In other MVP style apps where performance is not critical / load is very small, this way of working allows you to have a system in place where you can send things to head or footer without little hassle and is highly maintainable, more reasoning of the component stays in the component.

Then there is building and maintaining re-usable components where all things you need to manage are in the one file. For example a combobox we use a lot and share amongst projects would benefit the push stack to make it more 'plug and play'.

So all fair points and there of course is not one single way.

Benchmarks:

An html page with 500 div tags, with push stacks tags present:

-----
test with push stacks middleware + no caching: 1.75 ms
-----
test with push stacks middleware + django middleware caching: 0.88 ms
-----
test without push stacks middleware (feature inactive): 0.78 ms

So, we see an approximate 0.1ms difference between loading the page without push stacks enabled vs cached middleware. For me this is of course negligible.

The auto-config could append the push stack middleware wrapped in Django's cached middlewares. An enhancement we could add later is to optionally handle minification in the middleware, which may bring better speed vs default!

1 reply

mattbha Oct 10, 2024

seems minimal impact! I imagine this middleware would be optional too.

mlissner · 2025-02-11T06:32:40Z

mlissner
Feb 11, 2025

It will not permit duplicate pushes to the same stack, so you can safely define this in your component and use the component multiple times, it will only display the one instance

I'm curious how "duplicate" is being defined and calculated. I'm assuming that I can push multiple times to the same stack, and so long as the pushes are different, they'll both work?

3 replies

wrabit Feb 11, 2025
Maintainer Author

@mlissner It would be just maintaining pushes that have unique keys and values. So you can have multiple push keys, like pushing something to the head area, but it will not put the same thing in the head more than once. It's really to support the idea of creating re-usable components that have a js area. But it could have other use cases as well, think teleport for vue and alpine or portals for react. And the 'unique' behaviour could also be configurable, i.e. there may be a time you want to send duplicate values to the same stack.

mlissner Feb 11, 2025

Phenomenal. I love the idea that you can have JS in the component without having to remember to add it to the page or worrying about adding it to the page multiple times.

mattbha Feb 11, 2025

I use alpinejs with cotton and its a match made in heaven. can already inline the js in the components.

the only negative is if you have the same component repeated many times, the js logic is duplicated. but its served gzipped to the browser so its not a big deal. in those cases, i could use this feature to define the js behavior once and have all (say cells of a table) reference it

fbinz · 2025-02-12T13:56:46Z

fbinz
Feb 12, 2025

I love the idea! It would fit quite well into the way I'm using django-cotton currently.

The name (bike-shedding...)

The name "stack" suggests some kind of ordering and push/pop semantics. In this case, it seems to be more like an append-only list. Maybe the term "Portal" (coming from React) would also fit?

Deduplication

This approach has the opinion that by default we should assume pushes to the same stack are made unique. There may be cases where duplicates are desired, this could be dealt with later with a 'multiple' attribute on the stack or push.

Maybe the responsibility wrt deduplication could (and should?) be delegated to the user? For example an explicit "id" parameter for the <c-push to="head" id="my-component" /> would seem quite sensible. The stack would then simply drop pushes, if the ID is already present.
This would allow multiple components to i.e. push a shared JS dependency to a stack.

Reducing the available component namespace

While two additional names (c-stack and c-push) seem fine, maybe we could just use <c-stack> with different arguments?
I.e.

<c-stack name="head" /> declares a new stack (theoretically, it could also act as a fallback, i.e. only declare a new stack if none of the given name exists).
<c-stack push="head" /> pushes to the "head" stack.

5 replies

wrabit Feb 12, 2025
Maintainer Author

@fbinz thanks for the feedback

Deduplication - Agree that it should be configurable. About ID, I'd rather not have to think of and maintain an additional naming system for this purpose (after all, naming is hard), if we can achieve it without then it would be simpler dx. Something like this could work:

<c-portal to="footerjs" unique>
  <script> // </script>  
</c-portal>

And we can still allow a single portal to display both types of snippets (unique and non-unique).

Preserving Namespace & Naming - Generally agree and like the idea of sticking to one new tag. I originally thought of teleport but on seconds thoughts - it's difficult to come up with a suitable name for the attribute on the destination tag, but portal already seems to describe it better and it can just be:

<c-portal name="head" />

fbinz Feb 12, 2025

<c-portal to="footerjs" unique>
  <script> // </script>  
</c-portal>

Just to understand correctly: Is this supposed to be based on the contents of the <c-portal> tag?
I.e. having two components, each with a <c-portal to="footerjs" unique>ABC</c-portal> would result in a single ABC rendered in the footerjs portal?

And a third <c-portal to="footerjs" unique>DEF</c-portal> would result in ABCDEF (or DEFABC for that matter...)?

If so, I think that could be a nice solution, though I wonder how to deal with white space or other cosmetic markup differences in the portal contents.

wrabit Feb 12, 2025
Maintainer Author

Correct and the contents of the portal will be rendered as it was captured, new lines, indentation and all.

fbinz Feb 13, 2025

Understood. It sounds a little brittle, because a single white space could have unintended consequences, but maybe in practice it won't be an issue.

wrabit Feb 13, 2025
Maintainer Author

Yeah mostly I think it will be used for sending <script>, <link> and general html tags where whitespace will not matter, but we can tweak based on any raised issues.

Push Stacks for Better Asset Management + More #183

wrabit Oct 6, 2024 Maintainer

The problem

The current solution

push/stack pattern:

Features notes

How it works:

Further enhancements

Concerns

Replies: 8 comments · 16 replies

wrabit Oct 10, 2024 Maintainer Author

wrabit Oct 10, 2024 Maintainer Author

my_view.html

button.html

tooltip.html

wrabit Oct 10, 2024 Maintainer Author

wrabit Oct 10, 2024 Maintainer Author

wrabit Oct 10, 2024 Maintainer Author

wrabit Feb 11, 2025 Maintainer Author

The name (bike-shedding...)

Deduplication

Reducing the available component namespace

wrabit Feb 12, 2025 Maintainer Author

wrabit Feb 12, 2025 Maintainer Author

wrabit Feb 13, 2025 Maintainer Author

wrabit
Oct 6, 2024
Maintainer

Replies: 8 comments 16 replies

wrabit Oct 10, 2024
Maintainer Author

wrabit Oct 10, 2024
Maintainer Author

wrabit Oct 10, 2024
Maintainer Author

wrabit Oct 10, 2024
Maintainer Author

wrabit
Oct 10, 2024
Maintainer Author

wrabit Feb 11, 2025
Maintainer Author

wrabit Feb 12, 2025
Maintainer Author

wrabit Feb 12, 2025
Maintainer Author

wrabit Feb 13, 2025
Maintainer Author