MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation
Paper
•
2502.04299
•
Published
•
18
Nikki Russell PRO
Dragunflie-420
AI & ML interests
None yet
Recent Activity
upvoted
a
paper
29 days ago
MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video
Generation
upvoted
a
paper
29 days ago
A Picture is Worth a Thousand Words: Principled Recaptioning Improves
Image Generation
liked
a model
29 days ago
stabilityai/stable-diffusion-xl-base-1.0
Organizations
Dragunflie-420's activity
upvoted
2
papers
29 days ago
liked
a
model
29 days ago
published
a
Space
about 1 month ago
liked
2
Spaces
about 2 months ago
updated
a
Space
about 2 months ago
upvoted
a
paper
about 2 months ago
reacted to
Severian's
post with 👍
about 2 months ago
Post
3927
Interesting Solution to the Problem of Misguided Attention
So I've been fascinated by the problem of Misguided Attention for a few weeks. I am trying to build an inference algorithm to help LLMs address that issue; but in the process, I found a cool short-term fix I call "Mindful Attention" using just prompt-engineering.
Have you ever thought about how our brains filter reality through layers of past experiences, concepts, and mental images? For example, when you look at an oak tree, are you truly seeing that oak tree in all its unique details, or are you overlaying it with a generalized idea of "oak tree"? This phenomenon inspired the new approach.
LLMs often fall into a similar trap, hence the Misguided Attention problem. They process input not as it’s uniquely presented but through patterns and templates they’ve seen before. This leads to responses that can feel "off," like missing the point of a carefully crafted prompt or defaulting to familiar but irrelevant solutions.
I wanted to address this head-on by encouraging LLMs to slow down, focus, and engage directly with the input—free of assumptions. This is the core of the Mindful Attention Directive, a prompt designed to steer models away from over-generalization and back into the moment.
You can read more about the broader issue here: https://github.com/cpldcpu/MisguidedAttention
And if you want to try this mindful approach in action, check out the LLM I’ve set up for testing: https://hf.co/chat/assistant/677e7ebcb0f26b87340f032e. It works about 80% of the time to counteract these issues, and the results are pretty cool.
I'll add the Gist with the full prompt. I admit, it is quite verbose but it's the most effective one I have landed on yet. I am working on a smaller version that can be appended to any System Prompt to harness the Mindful Attention. Feel free to experiment to find a better version for the community!
Here is the Gist: https://gist.github.com/severian42/6dd96a94e546a38642278aeb4537cfb3
So I've been fascinated by the problem of Misguided Attention for a few weeks. I am trying to build an inference algorithm to help LLMs address that issue; but in the process, I found a cool short-term fix I call "Mindful Attention" using just prompt-engineering.
Have you ever thought about how our brains filter reality through layers of past experiences, concepts, and mental images? For example, when you look at an oak tree, are you truly seeing that oak tree in all its unique details, or are you overlaying it with a generalized idea of "oak tree"? This phenomenon inspired the new approach.
LLMs often fall into a similar trap, hence the Misguided Attention problem. They process input not as it’s uniquely presented but through patterns and templates they’ve seen before. This leads to responses that can feel "off," like missing the point of a carefully crafted prompt or defaulting to familiar but irrelevant solutions.
I wanted to address this head-on by encouraging LLMs to slow down, focus, and engage directly with the input—free of assumptions. This is the core of the Mindful Attention Directive, a prompt designed to steer models away from over-generalization and back into the moment.
You can read more about the broader issue here: https://github.com/cpldcpu/MisguidedAttention
And if you want to try this mindful approach in action, check out the LLM I’ve set up for testing: https://hf.co/chat/assistant/677e7ebcb0f26b87340f032e. It works about 80% of the time to counteract these issues, and the results are pretty cool.
I'll add the Gist with the full prompt. I admit, it is quite verbose but it's the most effective one I have landed on yet. I am working on a smaller version that can be appended to any System Prompt to harness the Mindful Attention. Feel free to experiment to find a better version for the community!
Here is the Gist: https://gist.github.com/severian42/6dd96a94e546a38642278aeb4537cfb3
liked
a
Space
3 months ago
updated
a
collection
3 months ago
upvoted
a
collection
3 months ago
updated
a
model
4 months ago
liked
a
dataset
4 months ago
does not work
1
#4 opened 5 months ago
by
Dragunflie-420
upvoted
a
paper
5 months ago
updated
a
Space
5 months ago
You are correct. Theres always an amount of FREE to do when building and becoming successful.
In college I can remember the volunteer aspects of it. There was a particular course that was suggested across the board to garner your hours necessary for graduation. Most would find out they needed an internship or equivalent FREE aka volunteer hours to walk the stage and receive their certificate diplomas.
If that university gets nothing else right it got that right lol.
To keep it you have to give it away thats damn good advice and its FREE!!!
reacted to
clem's
post with 👍
5 months ago
Post
3718
Very few people realize that most of the successful AI startups got successful because they were focused on open science and open-source for at least their first few years. To name but a few, OpenAI (GPT, GPT2 was open-source), Runway & Stability (stable diffusion), Cohere, Mistral and of course Hugging Face!
The reasons are not just altruistic, it's also because sharing your science and your models pushes you to build AI faster (which is key in a fast-moving domain like AI), attracts the best scientists & engineers and generates much more visibility, usage and community contributions than if you were 100% closed-source. The same applies to big tech companies as we're seeing with Meta and Google!
More startups and companies should release research & open-source AI, it's not just good for the world but also increases their probability of success!
The reasons are not just altruistic, it's also because sharing your science and your models pushes you to build AI faster (which is key in a fast-moving domain like AI), attracts the best scientists & engineers and generates much more visibility, usage and community contributions than if you were 100% closed-source. The same applies to big tech companies as we're seeing with Meta and Google!
More startups and companies should release research & open-source AI, it's not just good for the world but also increases their probability of success!
-
- 4 replies
Add generated example
1
#1 opened 5 months ago
by
Dragunflie-420