3541 stories
·
2 followers

HTML attributes vs DOM properties

1 Share

Comments

Read the whole story
emrox
1 hour ago
reply
Hamburg, Germany
Share this story
Delete

VideoGigaGAN: Towards Detail-rich Video Super-Resolution

1 Share

🐢University of Maryland, College Park        Adobe Research

8× Upsampling results (128×128→1024×1024)

Our model is able to upsample a video up to 8× with rich details.

Abstract

Video super-resolution (VSR) approaches have shown impressive temporal consistency in upsampled videos. However, these approaches tend to generate blurrier results than their image counterparts as they are limited in their generative capability. This raises a fundamental question: can we extend the success of a generative image upsampler to the VSR task while preserving the temporal consistency? We introduce VideoGigaGAN, a new generative VSR model that can produce videos with high-frequency details and temporal consistency. VideoGigaGAN builds upon a large-scale image upsampler -- GigaGAN. Simply inflating GigaGAN to a video model by adding temporal modules produces severe temporal flickering. We identify several key issues and propose techniques that significantly improve the temporal consistency of upsampled videos. Our experiments show that, unlike previous VSR methods, VideoGigaGAN generates temporally consistent videos with more fine-grained appearance details. We validate the effectiveness of VideoGigaGAN by comparing it with state-of-the-art VSR models on public datasets and showcasing video results with 8× super-resolution.

Overview: Why is it challenging?

Method Overview

Our Video Super-Resolution (VSR) model is built upon the asymmetric U-Net architecture of the image GigaGAN upsampler. To enforce temporal consistency, we first inflate the image upsampler into a video upsampler by adding temporal attention layers into the decoder blocks. We also enhance consistency by incorporating the features from the flow-guided propagation module. To suppress aliasing artifacts, we use Anti-aliasing block in the downsampling layers of the encoder. Lastly, we directly shuttle the high frequency features via skip connection to the decoder layers to compensate for the loss of details in the BlurPool process.

Ablation study

Strong hallucination capability of image GigaGAN results in temporally flickering artifacts, especially aliasing caused by the artifacted LR input.

Slide to switch between different examples


We progressively add components to the base model to handle these artifacts →

Comparison with previous methods

Compared to previous models, our models provides a detail-rich result with comparable temporal consistency.


Results on generic videos (128×128→512×512)

Our model is able to handle generic videos of different categories.

BibTeX

@article{xu2024videogigagan,
      title={VideoGigaGAN: Towards Detail-rich Video Super-Resolution}, 
      author={Yiran Xu and Taesung Park and Richard Zhang and Yang Zhou and Eli Shechtman and Feng Liu and Jia-Bin Huang and Difan Liu},
      year={2024},
      eprint={2404.12388},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
  }
Read the whole story
emrox
2 hours ago
reply
Hamburg, Germany
Share this story
Delete

Why Cats Knock Stuff Over

1 Share

Why do cats like to push stuff over edges and then curiously watch the fallen object? I suggest that they are play-hunting, and are testing the ‘prey’ for liveness & playing-dead, similarly to tossing or poking it with claws.

Good cat toys sim­u­late hunt­ing.⁠ The play = hunt­ing par­a­digm may also ex­plains why cats love to push things over edges.

The ques­tion of why cats fa­mously knock ob­jects over is un­re­searched as far as I know: it’s mys­te­ri­ous, be­cause they often push over the same ob­ject as be­fore, so it doesn’t seem to be novel or learn­ing, one would think; but writ­ing it off as ‘bore­dom’ or ‘ran­dom­ness’ is not an an­swer, be­cause there are so many other ways to mod­ify an en­vi­ron­ment, and this pro­vides no ex­pla­na­tion for why knocking-over spe­cific ob­jects is so consistently-chosen be­hav­ior.

Why? Be­cause push­ing tests the pos­si­bil­ity of de­cep­tive prey playing-dead! To ex­plain it, one might sur­mise that ‘knock­ing over’ is an ex­plo­rative hunt­ing be­hav­ior, test­ing prey for in­for­ma­tion about whether it’s merely play­ing dead⁠.

This ex­plains why the ob­jects tend to be prey-like in size (with large or shat­ter­ing ob­jects being fright­en­ing), some­times semi-mobile, they watch the re­sults so in­tently de­spite the (to us) pre­dictabil­ity, and per­se­vere in it but de­creas­ingly so.

Q-tips have many uses. Like cleaning earwax from an ear, and then letting a cat nom it⁠, and then, oft as not, knock it off the table or desk—as indeed cats are notorious for doing with any small object in reach. But while watching my cat play with Q-tips (used or not), I’ve noticed that no matter how many times I put down a Q-tip on my desk after he’d knocked it off, he was still interested in knocking it off & then watching it intently—as if he had suddenly begun to doubt the law of gravity or suffered amnesia about the countless prior instances, and needed to test it again and again.

Read the whole story
emrox
1 day ago
reply
Hamburg, Germany
Share this story
Delete

Distinguished 2

2 Shares

Distinguished 2

So ends my unplanned Snake-With-A-Moustache Week!

(And this is a sequel to a moustache-laden comic from 10 years ago!)

Read the whole story
emrox
1 day ago
reply
Hamburg, Germany
ameel
11 days ago
reply
Melbourne, Australia
Share this story
Delete

Häagen-Bot

2 Shares

Häagen-Bot

And more robots.

Read the whole story
emrox
1 day ago
reply
Hamburg, Germany
ameel
4 days ago
reply
Melbourne, Australia
Share this story
Delete

useTransition – React

1 Share

useTransition is a React Hook that lets you update the state without blocking the UI.

const [isPending, startTransition] = useTransition()


Reference

useTransition()

Call useTransition at the top level of your component to mark some state updates as Transitions.

import { useTransition } from 'react';

function TabContainer() {

const [isPending, startTransition] = useTransition();

// ...

}

See more examples below.

Parameters

useTransition does not take any parameters.

Returns

useTransition returns an array with exactly two items:

  1. The isPending flag that tells you whether there is a pending Transition.
  2. The startTransition function that lets you mark a state update as a Transition.

startTransition function

The startTransition function returned by useTransition lets you mark a state update as a Transition.

function TabContainer() {

const [isPending, startTransition] = useTransition();

const [tab, setTab] = useState('about');

function selectTab(nextTab) {

startTransition(() => {

setTab(nextTab);

});

}

// ...

}

Parameters

Returns

startTransition does not return anything.

Caveats

  • useTransition is a Hook, so it can only be called inside components or custom Hooks. If you need to start a Transition somewhere else (for example, from a data library), call the standalone startTransition instead.

  • You can wrap an update into a Transition only if you have access to the set function of that state. If you want to start a Transition in response to some prop or a custom Hook value, try useDeferredValue instead.

  • The function you pass to startTransition must be synchronous. React immediately executes this function, marking all state updates that happen while it executes as Transitions. If you try to perform more state updates later (for example, in a timeout), they won’t be marked as Transitions.

  • A state update marked as a Transition will be interrupted by other state updates. For example, if you update a chart component inside a Transition, but then start typing into an input while the chart is in the middle of a re-render, React will restart the rendering work on the chart component after handling the input update.

  • Transition updates can’t be used to control text inputs.

  • If there are multiple ongoing Transitions, React currently batches them together. This is a limitation that will likely be removed in a future release.


Usage

Marking a state update as a non-blocking Transition

Call useTransition at the top level of your component to mark state updates as non-blocking Transitions.

import { useState, useTransition } from 'react';

function TabContainer() {

const [isPending, startTransition] = useTransition();

// ...

}

useTransition returns an array with exactly two items:

  1. The isPending flag that tells you whether there is a pending Transition.
  2. The startTransition function that lets you mark a state update as a Transition.

You can then mark a state update as a Transition like this:

function TabContainer() {

const [isPending, startTransition] = useTransition();

const [tab, setTab] = useState('about');

function selectTab(nextTab) {

startTransition(() => {

setTab(nextTab);

});

}

// ...

}

Transitions let you keep the user interface updates responsive even on slow devices.

With a Transition, your UI stays responsive in the middle of a re-render. For example, if the user clicks a tab but then change their mind and click another tab, they can do that without waiting for the first re-render to finish.

The difference between useTransition and regular state updates

Example 2 of 2:

Updating the current tab without a Transition

In this example, the “Posts” tab is also artificially slowed down so that it takes at least a second to render. Unlike in the previous example, this state update is not a Transition.

Click “Posts” and then immediately click “Contact”. Notice that the app freezes while rendering the slowed down tab, and the UI becomes unresponsive. This state update is not a Transition, so a slow re-render freezed the user interface.

[ARTIFICIALLY SLOW] Rendering 500 <SlowPost />

[ARTIFICIALLY SLOW] Rendering 500 <SlowPost />


Updating the parent component in a Transition

You can update a parent component’s state from the useTransition call, too. For example, this TabButton component wraps its onClick logic in a Transition:

export default function TabButton({ children, isActive, onClick }) {

const [isPending, startTransition] = useTransition();

if (isActive) {

return <b>{children}</b>

}

return (

<button onClick={() => {

startTransition(() => {

onClick();

});

}}>

{children}

</button>

);

}

Because the parent component updates its state inside the onClick event handler, that state update gets marked as a Transition. This is why, like in the earlier example, you can click on “Posts” and then immediately click “Contact”. Updating the selected tab is marked as a Transition, so it does not block user interactions.

[ARTIFICIALLY SLOW] Rendering 500 <SlowPost />

[ARTIFICIALLY SLOW] Rendering 500 <SlowPost />


Displaying a pending visual state during the Transition

You can use the isPending boolean value returned by useTransition to indicate to the user that a Transition is in progress. For example, the tab button can have a special “pending” visual state:

function TabButton({ children, isActive, onClick }) {

const [isPending, startTransition] = useTransition();

// ...

if (isPending) {

return <b className="pending">{children}</b>;

}

// ...

Notice how clicking “Posts” now feels more responsive because the tab button itself updates right away:


Preventing unwanted loading indicators

In this example, the PostsTab component fetches some data using a Suspense-enabled data source. When you click the “Posts” tab, the PostsTab component suspends, causing the closest loading fallback to appear:

Hiding the entire tab container to show a loading indicator leads to a jarring user experience. If you add useTransition to TabButton, you can instead indicate display the pending state in the tab button instead.

Notice that clicking “Posts” no longer replaces the entire tab container with a spinner:

Read more about using Transitions with Suspense.

Note

Transitions will only “wait” long enough to avoid hiding already revealed content (like the tab container). If the Posts tab had a nested <Suspense> boundary, the Transition would not “wait” for it.


Building a Suspense-enabled router

If you’re building a React framework or a router, we recommend marking page navigations as Transitions.

function Router() {

const [page, setPage] = useState('/');

const [isPending, startTransition] = useTransition();

function navigate(url) {

startTransition(() => {

setPage(url);

});

}

// ...

This is recommended for two reasons:

Here is a tiny simplified router example using Transitions for navigations.

Note

Suspense-enabled routers are expected to wrap the navigation updates into Transitions by default.


Displaying an error to users with an error boundary

Canary

Error Boundary for useTransition is currently only available in React’s canary and experimental channels. Learn more about React’s release channels here.

If a function passed to startTransition throws an error, you can display an error to your user with an error boundary. To use an error boundary, wrap the component where you are calling the useTransition in an error boundary. Once the function passed to startTransition errors, the fallback for the error boundary will be displayed.

import { useTransition } from "react";
import { ErrorBoundary } from "react-error-boundary";

export function AddCommentContainer() {
  return (
    <ErrorBoundary fallback={<p>⚠️Something went wrong</p>}>
      <AddCommentButton />
    </ErrorBoundary>
  );
}

function addComment(comment) {
  
  if (comment == null) {
    throw new Error("Example Error: An error thrown to trigger error boundary");
  }
}

function AddCommentButton() {
  const [pending, startTransition] = useTransition();

  return (
    <button
      disabled={pending}
      onClick={() => {
        startTransition(() => {
          
          
          addComment();
        });
      }}
    >
      Add comment
    </button>
  );
}


Troubleshooting

Updating an input in a Transition doesn’t work

You can’t use a Transition for a state variable that controls an input:

const [text, setText] = useState('');

// ...

function handleChange(e) {

// ❌ Can't use Transitions for controlled input state

startTransition(() => {

setText(e.target.value);

});

}

// ...

return <input value={text} onChange={handleChange} />;

This is because Transitions are non-blocking, but updating an input in response to the change event should happen synchronously. If you want to run a Transition in response to typing, you have two options:

  1. You can declare two separate state variables: one for the input state (which always updates synchronously), and one that you will update in a Transition. This lets you control the input using the synchronous state, and pass the Transition state variable (which will “lag behind” the input) to the rest of your rendering logic.
  2. Alternatively, you can have one state variable, and add useDeferredValue which will “lag behind” the real value. It will trigger non-blocking re-renders to “catch up” with the new value automatically.

React doesn’t treat my state update as a Transition

When you wrap a state update in a Transition, make sure that it happens during the startTransition call:

startTransition(() => {

// ✅ Setting state *during* startTransition call

setPage('/about');

});

The function you pass to startTransition must be synchronous.

You can’t mark an update as a Transition like this:

startTransition(() => {

// ❌ Setting state *after* startTransition call

setTimeout(() => {

setPage('/about');

}, 1000);

});

Instead, you could do this:

setTimeout(() => {

startTransition(() => {

// ✅ Setting state *during* startTransition call

setPage('/about');

});

}, 1000);

Similarly, you can’t mark an update as a Transition like this:

startTransition(async () => {

await someAsyncFunction();

// ❌ Setting state *after* startTransition call

setPage('/about');

});

However, this works instead:

await someAsyncFunction();

startTransition(() => {

// ✅ Setting state *during* startTransition call

setPage('/about');

});


I want to call useTransition from outside a component

You can’t call useTransition outside a component because it’s a Hook. In this case, use the standalone startTransition method instead. It works the same way, but it doesn’t provide the isPending indicator.


The function I pass to startTransition executes immediately

If you run this code, it will print 1, 2, 3:

console.log(1);

startTransition(() => {

console.log(2);

setPage('/about');

});

console.log(3);

It is expected to print 1, 2, 3. The function you pass to startTransition does not get delayed. Unlike with the browser setTimeout, it does not run the callback later. React executes your function immediately, but any state updates scheduled while it is running are marked as Transitions. You can imagine that it works like this:

// A simplified version of how React works

let isInsideTransition = false;

function startTransition(scope) {

isInsideTransition = true;

scope();

isInsideTransition = false;

}

function setState() {

if (isInsideTransition) {

// ... schedule a Transition state update ...

} else {

// ... schedule an urgent state update ...

}

}

Read the whole story
emrox
2 days ago
reply
Hamburg, Germany
Share this story
Delete
Next Page of Stories