Dotnet Background Jobs With Hangfire

Most of the applications needs to implement background jobs or scheduled jobs to accomplish application requirements such as email remainders, data processing, recurring automated reports, and report generation for large data, database maintenance, mass notification/email/sms ques and processing application cache reset to keep the data in consistent state.

There are lots of ways to run a background job, but sometimes .net code which need to be run in the background because running it on a main thread is not appropriate, that’s can be either due to a specific requirement or due to its nature. So this post is to discuss about Hangfire and its implementations in .net core project.

Hangfire

Hangfire is an open-source framework that gives an easy way to perform background jobs in .net and .net core applications. It’s helps to execute/schedule and manage background jobs in an reliable way. It’s backed by persistent storage such as SQL Server, SQL, Azure, Redis etc.

Why you need a framework for background task and why can't do it by yourself?

ASP.net Issues

1. AppDomain and App Pool recycling

2. AppDomain and App Pool recycling so it’s difficult to keep the thread in a consistent state

3. When you modify web.config, ASP.NET will recycle the AppDomain, though the w3wp.exe process (the IIS web server process) stays alive.

4. IIS will itself recycle the entire w3wp.exe process every 29 hours.

5. In a shared hosting environment, many web servers are configured to tear down the application pools after some period of inactivity. For example, if there are no requests to the application within a 20 minute period, it may take down the app domain. Or if we exceed the allotted limit or something then the some hosting providers will automatically tear down the pool.

6. Multiple Instances and load of Main Thread

Features of Hangfire

1. It’s simple to use.

2. Support for all kind of background task – short and long running.

3. CPU Intensive and I/O Intensive and ready to use

4. Fire and forget or schedule tasks for particular interval of time.

5. Reliability of jobs

6. Managed by its own dashboard

7. Backed by persistent storage

8. Automatic retires (When exceptions or app shutdown or unexpected process terminations)

9. Can be distributed or simple

HangFire - All type of jobs

1. Fire and Forget jobs

2. Delayed jobs

3. Recurring jobs

4. Continuation jobs

5. Batch jobs

6. Batch continues jobs

Hangfire Architecture

Hangfire Architecture

Fig ref: (https://docs.hangfire.io/en/latest/_images/hangfire-workflow.png)

Client can create any kind of background jobs and these job information stored in persistent storage (SQL Server, SQL, Azure, Redis etc). And this persistent storage helps background jobs to survive on application restarts, server reboots etc.

Background jobs are processed by Hangfire Server. It is implemented as a set of dedicated (not thread pool’s) background threads that fetch jobs from a storage and process them. Server is also responsible to keep the storage clean and remove old data automatically.

Hangfire Implementation In .Net core

1. Install Nuget Package

Install-Package HangFire

2. Startup.cs

2.1 Add the below line in ConfigureServices(IServiceCollection services) method

 // Hangfire DB and other Configuration
 services.AddHangfire(
 x => x.UseSqlServerStorage(Configuration.GetConnectionString("KodeOverflowDemo_HFDemoDB"))
 );
 services.AddHangfireServer();

2.2 Add the below one in Configure(IApplicationBuilder app, IWebHostEnvironment env)

 app.UseHangfireDashboard(); // to use hangfire dashboard ui
 
// to start a job as soon as application launches add the below line just after app.UseHangfireDashboard();
 
 BackgroundJob.Enqueue(() => Console.WriteLine("Greetings from Hangfire Demo By KodeOverflow!")); 

3. Add connection string to you appsetting.json

"ConnectionStrings": {
 "DefaultConnection": "Data Source=.\\SQLEXPRESS2014;Initial Catalog=KodeOverflowDemo_HFDemoDB;Integrated Security=True;"
 }

Note: Here I’m using SQL Server as hangfire persistent database

If you have existing application with any of the persistent database supported by Hangfire, then you can use that or you can setup a separate DB for Hangfire.

Also note that, Hangfire will look for configured database and check the required schema, If the schema doesn’t exist then it will create the schema. Here is the sample database of my demo. However you don't need to worry about these schema, hangfire will take care of that 😉.

After these setup, now you are ready to go, let’s run it 😍

Fig: Hangfire DB schema

Hangfire offers Dashboard UI, that offers management of background tasks, monitoring etc.

Hangfire Dashboard UI is available In url http://<website-url>/hangfire, Hangfire is the default path of the hangfire Dashboard UI.

Let's run and see what happens. 😋

HangFire Dashboard UI.

Job tab of hangfire dashboard UI

You can view the successfully completed task under Jobs -> Succeeded

Succeed jobs list of Hangfire demo

You can also view the job details by clicking on the job which is listed on that table

Job details screen of hangfire

Executing background task from controller.

You can kick off your background job from anywhere in the .net core project, Can be from start.cs, or from controller, or it can be from a service layer. However the below example shows the execution of background task from controller.

HomeController.cs

Different types of background jobs

1. Fire and Forget

These jobs execute only once and almost immediately after fired.

  var id = BackgroundJob.Enqueue(() => Console.WriteLine("Fire and forget"));

2. Recurring

Recurring jobs are fired many times on the specified interval (CRON Schedule),

RecurringJob.AddOrUpdate(() => DoeSomeReccuringTask(), Cron.Daily);

3. Delayed

Executed only once, but executed only after the specified time interval.

 var jobId = BackgroundJob.Schedule(() => Console.WriteLine("Delayed Job!"),TimeSpan.FromDays(7)); 

4. Continuations

These tasks are executed when its parent job is finished.

var id = BackgroundJob.Enqueue(() => DoSomeTask());

// to start the second task after the completion of first one. The second task will be in ques until the completion of first task

 BackgroundJob.ContinueJobWith(id, () => DoSomeOtherTask());

Here id is the jobid of parent job, so the second task executed only when the first one completed its execution.

5. Batched

Batch is a group of background jobs created atomically

var batchId = Batch.StartNew(x =>
{
 x.Enqueue(() => Console.WriteLine("Job 1"));
 x.Enqueue(() => Console.WriteLine("Job 2"));
});

6. Batch Continuations

Batch continuation is fired after all background jobs in a parent batch have finished.

Batch.ContinueBatchWith(batchId, x =>
{
 x.Enqueue(() => Console.WriteLine("Last Job"));
});

Note: batch is available for Hangfire pro version.

Customizing Hangfire Dashboard and Remote access

By default hangfire url is setup to http://<website-url>/hangfire, you can define your own custom url by passing the url string like this

app.UseHangfireDashboard("/dashboard"); 

By default hangfire Dashboard UI is only available on localhost, that’s due to the security reasons. However you can make is available to the remote too by adding authentication filters.

  app.UseHangfireDashboard("/KodeOverflowHangFireJobs", new 
  DashboardOptions
 {
 Authorization = new[] { new HangFireAuthorizationFilter() },
  IsReadOnlyFunc = (DashboardContext context) => true
 });

HangFireAuthorizationFilter.cs and StartUp.cs

All is set and now it’s time to do it by yourself 😜.

Happy coding 😍

You can find the detailed documentation from the official Hangfire website.

Full source code @Github