dlukez - DataLoader 0.1.3-ci0001
A port of Facebook's DataLoader for .NET
PM> Install-Package DataLoader -Version 0.1.3-ci0001 -Source https://www.myget.org/F/dlukez/api/v3/index.json
> nuget.exe install DataLoader -Version 0.1.3-ci0001 -Source https://www.myget.org/F/dlukez/api/v3/index.json
> dotnet add package DataLoader --version 0.1.3-ci0001 --source https://www.myget.org/F/dlukez/api/v3/index.json
source https://www.myget.org/F/dlukez/api/v3/index.json
nuget DataLoader ~> 0.1.3-ci0001
Copy to clipboard
> choco install DataLoader --version 0.1.3-ci0001 --source https://www.myget.org/F/dlukez/api/v2
Import-Module PowerShellGet
Register-PSRepository -Name "dlukez" -SourceLocation "https://www.myget.org/F/dlukez/api/v2"
Install-Module -Name "DataLoader" -RequiredVersion "0.1.3-ci0001" -Repository "dlukez" -AllowPreRelease
Copy to clipboard
DataLoader for .NET
A port of Facebook's DataLoader for .NET.
This project began as a solution to the select N+1 problem for GraphQL .NET but was implemented as a standalone package that is completely decoupled from any framework.
It leverages .NET's async/await feature to enable query batching (a la Facebook's Dataloader) that should work out of the box, without requiring significant changes to an existing codebase.
If anyone finds this useful outside of GraphQL, feel free to drop me a message - I'm interested to know of other potential applications that could be catered to.
Check out the sample to see it used in a GraphQL implementation.
Caveats
Facebook's implementation runs in Javascript and takes advantage of the event loop to fire any pending requests for ID's collected during the previous frame. Unfortunately, not all .NET applications run in an event loop.
As such, we have defined a special frame or context to contain our load operations. Whenever we want to use a loader, we should be inside one of these contexts. A simple way to do this is by calling the static DataLoaderContext.Run
method. This method takes a user-supplied delegate and runs it in within a new context, before actually executing any loaders that were called within it.
Usage
There are two ways loaders can be used.
Method 1: Bound/explicit context (recommended)
With this approach, loader instances are obtained for a particular context using the context's GetDataLoader
methods. Along with the user-supplied fetch callback, these methods also take a key for caching and reusing instances.
var results = await DataLoaderContext.Run(async loadCtx =>
{
// Here we obtain a loader using the context's factory method.
var droidLoader = loadCtx.Factory.GetDataLoader<int, Droid>("droids", ids =>
{
using (var db = new StarWarsContext())
{
return db.Droid.Where(d => ids.Contains(d.Id)).ToListAsync();
}
});
// Queue up some loads.
var task1 = droidLoader.LoadAsync(1);
var task2 = droidLoader.LoadAsync(2);
var task3 = droidLoader.LoadAsync(3);
// Await the results... Control is yielded to the framework and the loader is fired.
var results = Task.WhenAll(task1, task2, task3);
// We have the results, but let's load some more! Run ensures that asynchronous
// continuations behave like the initial call - ID's should be collected and
// fetched as a batch after continuations have run.
var task4 = droidLoader.LoadAsync(4);
var task5 = droidLoader.LoadAsync(5);
// Return all our results.
return (await Task.WhenAll(task4, task5)).Concat(results);
));
Example 2: Unbound/implicit context
// Create a floating/unbound loader that will attach itself to the context
// that's currently active at the time a load method is called.
var personLoader = new DataLoader<int, Person>(ids =>
{
using (var db = new StarWarsContext())
{
return db.Person.Where(p => ids.Contains(p.Id)).ToListAsync();
}
});
var results = await DataLoaderContext.Run(async () =>
{
// We have an implicit context here.
Debug.Assert(DataLoaderContext.Current != null);
// Queue up some person loads.
var task1 = personLoader.LoadAsync(1);
var task2 = personLoader.LoadAsync(2);
var task3 = personLoader.LoadAsync(3);
// Await the results... Control is yielded to the framework and the loader is fired.
var results = await Task.WhenAll(task1, task2, task3);
// We have the results, but let's load some more! Run ensures that asynchronous
// continuations behave like the initial call - ID's should be collected and
// fetched as a batch after continuations have run.
var task4 = personLoader.LoadAsync(4);
var task5 = personLoader.LoadAsync(5);
// Return all our results.
return (await Task.WhenAll(task4, task5)).Concat(results);
});
To Do
- Basic support
- Support async fetching
- Cancellation
- Benchmarks
- Multithreaded performance
Ideas
- Single worker thread to service loaders
- Sync context to handle async/await in load continuations
- .NETStandard 1.3: 1.3.0.0
OwnersDaniel |
AuthorsDaniel Zimmermann |
Project URLhttps://github.com/dlukez/dataloader-dotnet |
LicenseMIT |
Tagsdataloader batch future |
Info148 total downloads |
5 downloads for version 0.1.3-ci0001 |
Download (10.73 KB) |
Found on the current feed only |
Package history
Version | Size | Last updated | Downloads | Mirrored? | |||
---|---|---|---|---|---|---|---|
1.0.0-CI00043 | 14.64 KB | Tue, 29 May 2018 09:11:54 GMT | 8 | ||||
0.3.0 | 12.69 KB | Sat, 23 Sep 2017 12:17:19 GMT | 7 | ||||
0.2.1-ci0004 | 12.7 KB | Mon, 18 Sep 2017 17:35:41 GMT | 7 | ||||
0.2.1-ci0001 | 27.48 KB | Tue, 13 Jun 2017 02:50:51 GMT | 7 | ||||
0.2.0 | 26.89 KB | Wed, 07 Jun 2017 18:11:46 GMT | 7 | ||||
0.1.6-ci0008 | 26.9 KB | Wed, 07 Jun 2017 17:59:12 GMT | 8 | ||||
0.1.6-ci0007 | 24.71 KB | Mon, 29 May 2017 23:52:26 GMT | 7 | ||||
0.1.6-ci0006 | 24.72 KB | Mon, 29 May 2017 17:44:15 GMT | 8 | ||||
0.1.6-ci0005 | 26.59 KB | Mon, 29 May 2017 06:43:29 GMT | 6 | ||||
0.1.5 | 27.06 KB | Tue, 11 Apr 2017 05:50:12 GMT | 8 | ||||
0.1.5-ci0004 | 27.07 KB | Mon, 06 Mar 2017 16:53:23 GMT | 7 | ||||
0.1.5-ci0003 | 27.3 KB | Mon, 20 Feb 2017 01:52:40 GMT | 7 | ||||
0.1.5-ci0002 | 27.31 KB | Mon, 20 Feb 2017 00:34:06 GMT | 7 | ||||
0.1.4-ci0002 | 27.3 KB | Fri, 17 Feb 2017 12:43:19 GMT | 5 | ||||
0.1.4-ci0001 | 27.33 KB | Fri, 17 Feb 2017 06:16:04 GMT | 7 | ||||
0.1.3 | 18.85 KB | Wed, 15 Feb 2017 19:27:39 GMT | 7 | ||||
0.1.3-ci0012 | 18.85 KB | Wed, 15 Feb 2017 17:51:40 GMT | 7 | ||||
0.1.3-ci0009 | 18.85 KB | Wed, 15 Feb 2017 17:25:03 GMT | 6 | ||||
0.1.3-ci0002 | 10.74 KB | Mon, 13 Feb 2017 04:42:01 GMT | 6 | ||||
0.1.3-ci0001 | 10.73 KB | Sun, 12 Feb 2017 20:23:22 GMT | 5 | ||||
0.1.2 | 10.72 KB | Sun, 12 Feb 2017 16:58:23 GMT | 6 | ||||
0.1.2-ci0073 | 10.74 KB | Sun, 12 Feb 2017 10:14:59 GMT | 5 |