# Long running processes

## Long running tasks

Long running tasks are ones that cannot be completed within the normal message processing window. These are tasks like encoding a video or preparing a large file for download. Because they take a long time to complete, they should be run in a way that doesn't block the message handling process.‌

When a message is read from the underlying queue it must be processed and deleted within a configured timeout (usually a few minutes). If it is not deleted in this time, the underlying queue will assume that the consuming process has died and will flag the message as visible again and will be picked up by another handler that will do the same process. After a number of iterations like this, the message will be sent to the dead letter queue.‌

The other downside to running time-consuming processes in a handling window is that it blocks that worker from consuming other messages as it's waiting for the process to complete. Handlers should process messages as fast as possible.‌

### Approaching long running tasks

Let's say we have a command **EncodeVideo**. This might take up to an hour to complete so a handler can't wait for it to complete without the message being returned to the queue.‌

Instead a separate process should be started, and the events that follow reflect the asynchronous nature of the task.‌

### A naive approach <a href="#a-naive-approach" id="a-naive-approach"></a>

A simple way that is **not recommended** might be to background the task:

```typescript
import { videoService } from 'services'​

export const encodeVideoHandler = (encodeVideo: EncodeVideo) => { 
  setTimeout(async () => videoService.encode(encodeVideo), 0)
}
```

This will background the task and the **EncodeVideo** command will be deleted, but it has a couple of flaws.‌

There is no retry mechanism at this point if the process fails. It will simply die without publishing any messages to indicate as such.‌

There's no effective load-balancing of tasks occurring. The same handler service may receive all of the **EncodeVideo** command and may attempt to have hundreds of these processes running in the background that eventually crash the instance.‌

Lastly if the service is restarted then the backgrounded tasks are killed and won't be retried.‌

### A containerised approach <a href="#a-containerised-approach" id="a-containerised-approach"></a>

If your app runs in kubernetes, docker swarm, ECS etc, then starting a pod/task per long running task can be very effective. This is outside the scope of what **@node-ts/bus** provides but can be implemented relatively simply with handlers.‌

When receiving an **EncodeVideo** command, use the handler to start the encoding process in a new pod/task and leave it up to the scheduler to place. This should also make it easier to scale out your app given the volume of these long running tasks being created.‌

#### Resiliency <a href="#resiliency" id="resiliency"></a>

Just using handlers is a good start, but it won't provide the reliability needed if a task fails or gets terminated by the scheduler.‌

​[Workflows](/guide/workflows.md) can be used to listen for [system messages](/guide/messages/system-messages.md) from the scheduler that indicate when a task as exited, and rerun the task if necessary.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://bus.node-ts.com/guide/long-running-processes.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.