Hazumi News

Hey HN! We’re Adish & Kyle from Mosaic (https://edit.mosaic.so, https://docs.mosaic.so/, https://mosaic.so). Mosaic lets you create and run your own multimodal video editing agents in a node-based canvas. It’s different from traditional video editing tools in two ways: (1) the user interface and (2) the visual intelligence built into our agent.

We were engineers at Tesla and one day had a fun idea to make a YouTube video of Cybertrucks in Palo Alto. We recorded hours of cars driving by, but got stuck on how to scrub through all this raw footage to edit it down to just the Cybertrucks.

We got frustrated trying to accomplish simple tasks in video editors like DaVinci Resolve and Adobe Premiere Pro. Features are hidden behind menus, buttons, and icons, and we often found ourselves Googling or asking ChatGPT how to do certain edits.

We thought that surely now, with multimodal AI, we could accelerate this process. Better yet, an AI video editor could automatically apply edits based off what it sees and hears in your video. The idea quickly snowballed and we began our side quest to build “Cursor for Video Editing”.

We put together a prototype and to our amazement, it was able to analyze and add text overlays based on what it saw or heard in the video. We could now automate our Cybertruck counting with a single chat prompt. That prototype is shown here: https://www.youtube.com/watch?v=GXr7q7Dl9X0.

After that, we spent a chunk of time building our own timeline-based video editor and making our multimodal copilot powerful and stateful. In natural language, we could now ask chat to help with AI asset generation, enhancements, searching through assets, and automatically applying edits like dynamic text overlays. That version is shown here: https://youtu.be/X4ki-QEwN40.

After talking to users though, we realized that the chat UX has limitations for video: (1) the longer the video, the more time it takes to process. Users have to wait too long between chat responses. (2) Users have set workflows that they use across video projects. Especially for people who have to produce a lot of content, the chat interface is a bottleneck rather than an accelerant.

That took us back to first principles to rethink what a “non-linear editor” really means. The result: a node-based canvas which enables you to create and run your own multimodal video editing agents. https://screen.studio/share/SP7DItVD.

Each tile in the canvas represents a video editing operation and is configurable, so you still have creative control. You can also branch and run edits in parallel, creating multiple variants from the same raw footage to A/B test different prompts, models, and workflows. In the canvas, you can see inline how your content evolves as the agent goes through each step.

The idea is that canvas will run your video editing on autopilot, and get you 80-90% of the way there. Then you can adjust and modify it in an inline timeline editor. We support exporting your timeline state out to traditional editing tools like DaVinci Resolve, Adobe Premiere Pro, and Final Cut Pro.

We’ve also used multimodal AI to build in visual understanding and intelligence. This gives our system a deep understanding of video concepts, emotions, actions, spoken word, light levels, shot types.

We’re doing a ton of additional processing in our pipeline, such as saliency analysis, audio analysis, and determining objects of significance—all to help guide the best edit. These are things that we as human editors internalize so deeply we may not think twice about it, but reverse-engineering the process to build it into the AI agent has been an interesting challenge.

Some of our analysis findings: Optimal Safe Rectangles: https://assets.frameapp.ai/mosaicresearchimage1.png Video Analysis: https://assets.frameapp.ai/mosaicresearchimage2.png Saliency Analysis: https://assets.frameapp.ai/mosaicresearchimage3.png Mean Movement Analysis: https://assets.frameapp.ai/mosaicresearchimage4.png

Use cases for editing include: - Removing bad takes or creating script-based cuts from videos / talking-heads - Repurposing longer-form videos into clips, shorts, and reels (e.g. podcasts, webinars, interviews) - Creating sizzle reels or montages from one or many input videos - Creating assembly edits and rough cuts from one or many input videos - Optimizing content for various social media platforms (reframing, captions, etc.) - Dubbing content with voice cloning and lip syncing.

We also support use cases for generating content such as motion graphic animations, cinematic captions, AI UGC content, adding contextual AI-generated B-Rolls to existing content, or modifying existing video footage (changing lighting, applying VFX).

Currently, our canvas can be used to build repeatable agentic workflows, but we’re working on a fully autonomous agent which will be able to do things like: style transfer using existing video content, define its own editing sequence / workflow without needing a canvas, do research and pull assets from web references, and so on.

You can try it today at https://edit.mosaic.so. You can sign up for free and get started playing with the interface by uploading videos, making workflows on the canvas, and editing them in the timeline editor. We do paywall node runs to help cover model costs. Our API docs are at https://docs.mosaic.so. We’d love to hear your feedback!

Hi HN! We're excited to release our VS Code/Cursor extension for marimo [1], an open-source, reactive Python notebook.

This extension provides a native experience for working with marimo notebooks, a long-requested feature that we’ve worked hard to get right.

An LSP-first architecture

The core of our extension is a marimo notebook language server (marimo-lsp [2]). As far as we know, it’s the first notebook runtime to take this approach. The Language Server Protocol (LSP) [3] offers a small but important set of notebook-related capabilities that we use for document and kernel syncing; everything else is handled through custom actions and messages.

By building on LSP, we aim to create a path to expose marimo capabilities in additional environments (beyond VS Code/Cursor). The notebook features in LSP are still limited, but as the protocol evolves, we’ll be able to shift more functionality out of the extension and into the language server, making it available to a wider range of editors and tools. For example, this could enable:

- structural edits to notebook documents (e.g., adding or removing cells) [4]

- editor hover information that reflects the live runtime values of variables

Deep uv integration with PEP 723

Because marimo notebooks are plain Python files, we adopt PEP 723-style inline metadata [5] to describe a notebook’s environment. Tools such as uv already support this format: they read the metadata block, build or update the corresponding environment, and run the script inside it.

The marimo CLI already integrates with uv in "sandbox" mode [6] to manage an isolated environment defined by PEP 723 metadata for a single notebook. In the extension, our uv-based “sandbox controller” manages multiple notebooks: each notebook gets its own isolated, cached environment. The controller keeps the environment aligned with the dependencies declared in the file and can update that metadata automatically when imports are missing.

uv normally syncs such environments whenever you run a script, ensuring it matches the dependencies declared in its metadata; we apply this concept at the cell level so the environment stays in sync whenever cells run. The same cached uv environment is reused if you run the notebook as a script via uv (e.g., uv run notebook.py).

—-------

This work has been a complete rewrite, and we're grateful to the community for early feedback. While VS Code and the LSP support a subset of notebook features, the ecosystem has been shaped heavily by Jupyter, and we’ve had to work around some assumptions baked into existing APIs. We’ve been coordinating with the VS Code team and hope our work can help broaden the conversation—pushing the LSP notebook model forward and making room for runtimes that aren’t Jupyter-based.

We'd love to hear your thoughts!

[1] https://marimo.io

[2] https://github.com/marimo-team/marimo-lsp

[3] https://microsoft.github.io/language-server-protocol/

[4] https://github.com/microsoft/vscode-languageserver-node/issu...

[5] https://peps.python.org/pep-0723/

[6] https://docs.marimo.io/guides/package_reproducibility/