The structure of a modern web application nowadays consists of one or more APIs and one or more different type of clients that consume those APIs. The thing is that despite the fact that those clients may required to support the same business logic, they have entirely different specifications in terms of CPU, memory or even network bandwidth usage. To make things more clearer and denote the problem we are going to solve on this post let’s assume your boss wants you to build an online solution to support the business of the company. The requirement is simple: clients should be able to view their data through an ASP.NET MVC Web application, a mobile oriented website build with AngularJS, some native (Android or IOS) or hybrid Ionic mobile applications and last but not least a desktop application.
Houston we have a problem
After being shocked on your boss’s requirements you gather yourself up and decide that you are going to build a single API and make all the clients consume it. Is this going to work? It depends on how you design it. In case you decide to serve the same data over the wire for all types of clients then boom! As long as your business logic and hence your ViewModels being served grow, the clients with less CPU and memory power such as the mobile applications, will probably start being slower, impacting the user’s experience (not to mention your boss’s temper). If you think about it, a list item in a mobile application usually renders about 5 properties so why even bother to request the entire viewmodel which may reach 25 properties? From my experience, I have seen hybrid applications built with Ionic getting slower or even crash when trying to render 250 items with 10-15 properties for each item. This means that you have to be more careful with your API and find a solution that will not cause those types of issues on your clients. A simple solution that you may come up to is to have different versions of your APIs for each client. This would mean that you would have different Action methods and viewmodels for the same resources. I would recommend you to avoid this approach because this would cause dramatically maintainability and testability issues. Versioning in APIs should be used for supporting new features, not different clients.
The solution
The best solution would be to allow your different clients to request the exact properties for the resources being served. But what do I mean by that? Let’s explain it with an example. Let’s assume that there is a resource uri /api/tracks/1 that returns a single track with the following JSON properties:
{ "TrackId": 1, "AlbumId": 1, "Bytes": 11170334, "Composer": "Angus Young, Malcolm Young, Brian Johnson", "GenreId": 1, "MediaTypeId": 1, "Milliseconds": 343719, "Name": "For Those About To Rock (We Salute You)", "UnitPrice": 0.99 }
What we would like to achieve is to allow the client to request for certain properties on that resource by making a request to /api/tracks/1?props=bytes,milliseconds,name:
{ "Bytes": 11170334, "Milliseconds": 343719, "Name": "For Those About To Rock (We Salute You)" }
What we gain with this approach is reduce the amount of data being served over the wire between the server and the client and hence the time for the request to be completed. More over the client’s CPU and memory usage isn’t overwhelmed which means better user experience and application performance. This approach may sounds easy to implement but it isn’t. The algorithm I created is based on parsing and traversing a Newtonsoft.Json.Linq.JToken representation of the viewmodel. We will discuss more on this algorithm when we reach the respective section of the post but for now let’s say that the approach should support nested or navigation properties as well. For example assuming that we have the resource /api/albums/22:
{ "AlbumId": 22, "ArtistName": "Caetano Veloso", "Title": "Sozinho Remix Ao Vivo", "Track": [ { "TrackId": 223, "AlbumId": 22, "Bytes": 14462072, "Composer": null, "GenreId": 7, "MediaTypeId": 1, "Milliseconds": 436636, "Name": "Sozinho (Hitmakers Classic Mix)", "UnitPrice": 0.99 }, { "TrackId": 224, "AlbumId": 22, "Bytes": 6455134, "Composer": null, "GenreId": 7, "MediaTypeId": 1, "Milliseconds": 195004, "Name": "Sozinho (Hitmakers Classic Radio Edit)", "UnitPrice": 0.99 } ] }
We need to be able to request certain properties for the nested Track property as well by making a request to /api/albums/22?props=artistname,title,track(trackid;bytes;name).
{ "ArtistName": "Caetano Veloso", "Title": "Sozinho Remix Ao Vivo", "Track": [ { "TrackId": 223, "Bytes": 14462072, "Name": "Sozinho (Hitmakers Classic Mix)" }, { "TrackId": 224, "Bytes": 6455134, "Name": "Sozinho (Hitmakers Classic Radio Edit)" } ] }
Awesome right? Let’s start building this multi-client supported API!
ASP.NET Core and Entity Framework Core
The project we are going to build here will be an ASP.NET MVC 6 application which means you need to have ASP.NET Core installed on your machine. In case though you don’t want to follow along that’s OK. The most important part of the post is the method that alternates the API’s response according to the request’s options. Because of the fact that we need a database to demonstrate the multi-client support of our API I decided to use the well known Chinook database which you can download for free here. Create the Chinook database in your SQL Server by running the Chinook_SqlServer_AutoIncrementPKs.sql file you will find in the downloaded .zip file. I found in this post the opportunity to show you how to initialize the entities using the Database First approach with Entity Framework Core. Start by creating a new Web Application named ShapingAPI by selecting the ASP.NET 5 empty template. Alter the project.json as follow and let Visual Studio 2015 restore the packages.
{ "version": "1.0.0-*", "compilationOptions": { "emitEntryPoint": true }, "dependencies": { "Microsoft.ApplicationInsights.AspNet": "1.0.0-rc1", "Microsoft.AspNet.IISPlatformHandler": "1.0.0-rc1-final", "Microsoft.AspNet.Mvc": "6.0.0-rc1-final", "Microsoft.AspNet.Server.Kestrel": "1.0.0-rc1-final", "Microsoft.AspNet.StaticFiles": "1.0.0-rc1-final", "Microsoft.Extensions.Configuration.FileProviderExtensions": "1.0.0-rc1-final", "Microsoft.Extensions.Configuration.Json": "1.0.0-rc1-final", "Microsoft.Extensions.Logging": "1.0.0-rc1-final", "Microsoft.Extensions.Logging.Console": "1.0.0-rc1-final", "Microsoft.Extensions.Logging.Debug": "1.0.0-rc1-final", "EntityFramework.MicrosoftSqlServer": "7.0.0-rc1-final", "EntityFramework.MicrosoftSqlServer.Design": "7.0.0-rc1-final", "EntityFramework.Commands": "7.0.0-rc1-final", "AutoMapper.Data": "1.0.0-beta1", "Newtonsoft.Json": "7.0.1" }, "commands": { "web": "Microsoft.AspNet.Server.Kestrel", "ef": "EntityFramework.Commands" }, "frameworks": { "dnx451": { }, "dnxcore50": { } }, "exclude": [ "wwwroot", "node_modules" ], "publishExclude": [ "**.user", "**.vspscc" ] }
I have highlighted the packages and commands to be used for generating the entities from the Chinook database. Now open a command prompt, navigate at the root of the application where the project.json exists and run the following command:
dnx ef dbcontext scaffold "Server=(localdb)\v11.0;Database=Chinook;Trusted_Connection=True;" EntityFramework.MicrosoftSqlServer --outputDir Entities
This will scaffold the Entities for you and a DbContext class as well.
You shouldn’t alter those entities by yourself. In my case I have commented out the connection string related line in the ChinookContext class.
protected override void OnConfiguring(DbContextOptionsBuilder options) { //options.UseSqlServer(@"Server=(localdb)\v11.0;Database=Chinook;Trusted_Connection=True;"); }
.. and add an appsettings.json file under the root of the application as follow:
{ "Logging": { "IncludeScopes": false, "LogLevel": { "Default": "Verbose", "System": "Information", "Microsoft": "Information" } }, "Data": { "ChinookConnection": { "ConnectionString": "Server=(localdb)\\v11.0;Database=Chinook;Trusted_Connection=True;MultipleActiveResultSets=true" } } }
Alter the connection string to reflect your database environment. Now switch to the Startup.cs file and change it as follow:
public class Startup { public Startup(IHostingEnvironment env) { // Set up configuration sources. var builder = new ConfigurationBuilder() .AddJsonFile("appsettings.json"); if (env.IsEnvironment("Development")) { // This will push telemetry data through Application Insights pipeline faster, allowing you to view results immediately. builder.AddApplicationInsightsSettings(developerMode: true); } builder.AddEnvironmentVariables(); Configuration = builder.Build().ReloadOnChanged("appsettings.json"); } public IConfigurationRoot Configuration { get; set; } // This method gets called by the runtime. Use this method to add services to the container public void ConfigureServices(IServiceCollection services) { // Add framework services. services.AddApplicationInsightsTelemetry(Configuration); services.AddEntityFramework() .AddSqlServer() .AddDbContext<ChinookContext>(options => options.UseSqlServer(Configuration["Data:ChinookConnection:ConnectionString"])); // Repositories services.AddScoped<IAlbumRepository, AlbumRepository>(); services.AddScoped<IArtistRepository, ArtistRepository>(); services.AddScoped<ICustomerRepository, CustomerRepository>(); services.AddScoped<IEmployeeRepository, EmployeeRepository>(); services.AddScoped<IGenreRepository, GenreRepository>(); services.AddScoped<IInvoiceLineRepository, InvoiceLineRepository>(); services.AddScoped<IInvoiceRepository, InvoiceRepository>(); services.AddScoped<IMediaTypeRepository, MediaTypeRepository>(); services.AddScoped<IPlaylistTrackRepository, PlaylistTrackRepository>(); services.AddScoped<IPlaylistRepository, PlaylistRepository>(); services.AddScoped<ITrackRepository, TrackRepository>(); services.AddMvc() .AddJsonOptions(options => { options.SerializerSettings.ReferenceLoopHandling = Newtonsoft.Json.ReferenceLoopHandling.Ignore; options.SerializerSettings.Formatting = Newtonsoft.Json.Formatting.Indented; options.SerializerSettings.ContractResolver = new CamelCasePropertyNamesContractResolver(); }); } // This method gets called by the runtime. Use this method to configure the HTTP request pipeline public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory) { loggerFactory.AddConsole(Configuration.GetSection("Logging")); loggerFactory.AddDebug(); app.UseIISPlatformHandler(); app.UseApplicationInsightsRequestTelemetry(); app.UseApplicationInsightsExceptionTelemetry(); app.UseStaticFiles(); AutoMapperConfiguration.Configure(); app.UseMvc(routes => { routes.MapRoute( name: "default", template: "{controller=Home}/{action=Index}/{id?}"); // Uncomment the following line to add a route for porting Web API 2 controllers. //routes.MapWebApiRoute("DefaultApi", "api/{controller}/{id?}"); }); } // Entry point for the application. public static void Main(string[] args) => WebApplication.Run<Startup>(args); }
We haven’t created the repositories yet neither the AutoMapperConfiguarion class. Let’s start with the repositories first. We will only add Get related functions since retrieving customized data is the main problem we are trying to solve. Add the the Infrastructure/Data/Repositories folder hierarchy under the root as shown in the picture below.
Add the following classes under the Infrastructure/Data folder.
public interface IRepository<TEntity> { #region READ TEntity Get(Expression<Func<TEntity, bool>> predicate); TEntity Get(Expression<Func<TEntity, bool>> predicate, params Expression<Func<TEntity, object>>[] includeProperties); IQueryable<TEntity> GetAll(); IQueryable<TEntity> GetAll(params Expression<Func<TEntity, object>>[] includeProperties); #endregion }
public abstract class Repository<TEntity> : IRepository<TEntity> where TEntity : class { protected DbContext _context; protected DbSet<TEntity> _dbSet; public Repository(DbContext context) { _context = context; _dbSet = _context.Set<TEntity>(); } #region READ public TEntity Get(Expression<Func<TEntity, bool>> predicate) { return _dbSet.FirstOrDefault(predicate); } public TEntity Get(Expression<Func<TEntity, bool>> predicate, params Expression<Func<TEntity, object>>[] includeProperties) { IQueryable<TEntity> query = _dbSet; if (includeProperties != null) foreach (var property in includeProperties) { query = query.Include(property); } return query.FirstOrDefault(predicate); } public IQueryable<TEntity> GetAll() { return _dbSet.AsQueryable<TEntity>(); } public IQueryable<TEntity> GetAll(params Expression<Func<TEntity, object>>[] includeProperties) { IQueryable<TEntity> query = _dbSet.AsQueryable<TEntity>(); if (includeProperties != null) foreach (var property in includeProperties) { query = query.Include(property); } return query; } #endregion }
Now add the following files under the Infrastructure/Data/Repositories folder.
namespace ShapingAPI.Infrastructure.Data.Repositories { public interface IAlbumRepository : IRepository<Album> { } public interface IArtistRepository : IRepository<Artist> { IEnumerable<Artist> LoadAll(); Artist Load(int artistId); } public interface ICustomerRepository : IRepository<Customer> { IEnumerable<Customer> LoadAll(); Customer Load(int customerId); } public interface IEmployeeRepository : IRepository<Employee> { } public interface IGenreRepository : IRepository<Genre> { } public interface IInvoiceLineRepository : IRepository<InvoiceLine> { } public interface IInvoiceRepository : IRepository<Invoice> { IEnumerable<Invoice> LoadAll(); Invoice Load(int invoiceId); } public interface IMediaTypeRepository : IRepository<MediaType> { } public interface IPlaylistTrackRepository : IRepository<PlaylistTrack> { } public interface IPlaylistRepository : IRepository<Playlist> { } public interface ITrackRepository : IRepository<Track> { } }
namespace ShapingAPI.Infrastructure.Data.Repositories { public class AlbumRepository : Repository<Album>, IAlbumRepository { public AlbumRepository(ChinookContext context) : base(context) { } } public class ArtistRepository : Repository<Artist>, IArtistRepository { public ArtistRepository(ChinookContext context) : base(context) { } public IEnumerable<Artist> LoadAll() { IQueryable<Artist> query = this._dbSet; query = query.Include(a => a.Album).ThenInclude(al => al.Track); return query.ToList(); } public Artist Load(int artistId) { IQueryable<Artist> query = this._dbSet; query = query.Include(a => a.Album).ThenInclude(al => al.Track); return query.FirstOrDefault(a => a.ArtistId == artistId); } } public class CustomerRepository : Repository<Customer>, ICustomerRepository { public CustomerRepository(ChinookContext context) : base(context) { } public IEnumerable<Customer> LoadAll() { IQueryable<Customer> query = this._dbSet; query = query.Include(c => c.Invoice).ThenInclude(i => i.InvoiceLine); return query.ToList(); } public Customer Load(int customerId) { IQueryable<Customer> query = this._dbSet; query = query.Include(c => c.Invoice).ThenInclude(i => i.InvoiceLine); return query.FirstOrDefault(c => c.CustomerId == customerId); } } public class EmployeeRepository : Repository<Employee>, IEmployeeRepository { public EmployeeRepository(ChinookContext context) : base(context) { } } public class GenreRepository : Repository<Genre>, IGenreRepository { public GenreRepository(ChinookContext context) : base(context) { } } public class InvoiceLineRepository : Repository<InvoiceLine>, IInvoiceLineRepository { public InvoiceLineRepository(ChinookContext context) : base(context) { } } public class InvoiceRepository : Repository<Invoice>, IInvoiceRepository { public InvoiceRepository(ChinookContext context) : base(context) { } public IEnumerable<Invoice> LoadAll() { IQueryable<Invoice> query = this._dbSet; query = query.Include(i => i.Customer).ThenInclude(c => c.Invoice); query = query.Include(i => i.InvoiceLine); return query.ToList(); } public Invoice Load(int invoiceId) { IQueryable<Invoice> query = this._dbSet; query = query.Include(i => i.Customer).ThenInclude(c => c.Invoice); query = query.Include(i => i.InvoiceLine); return query.FirstOrDefault(i => i.InvoiceId == invoiceId); } } public class MediaTypeRepository : Repository<MediaType>, IMediaTypeRepository { public MediaTypeRepository(ChinookContext context) : base(context) { } } public class PlaylistTrackRepository : Repository<PlaylistTrack>, IPlaylistTrackRepository { public PlaylistTrackRepository(ChinookContext context) : base(context) { } } public class PlaylistRepository : Repository<Playlist>, IPlaylistRepository { public PlaylistRepository(ChinookContext context) : base(context) { } } public class TrackRepository : Repository<Track>, ITrackRepository { public TrackRepository(ChinookContext context) : base(context) { } } }
Create a Mappings folder under Infrastructure and add the following classes:
public class AutoMapperConfiguration { public static void Configure() { Mapper.Initialize(x => { x.AddProfile<DomainToViewModelMappingProfile>(); }); } }
public class DomainToViewModelMappingProfile : Profile { protected override void Configure() { //Mapper.CreateMap<Track, TrackViewModel>(); //Mapper.CreateMap<Album, AlbumViewModel>() // .ForMember(vm => vm.ArtistName, map => map.MapFrom(a => a.Artist.Name)); //Mapper.CreateMap<Invoice, InvoiceViewModel>(); //Mapper.CreateMap<Artist, ArtistViewModel>(); //Mapper.CreateMap<Customer, CustomerViewModel>() // .ForMember(vm => vm.Address, map => map.MapFrom(c => new AddressViewModel() // { // Address = c.Address, // City = c.City, // Country = c.Country, // PostalCode = c.PostalCode, // State = c.State // })) // .ForMember(vm => vm.Contact, map => map.MapFrom(c => new ContactViewModel() // { // Email = c.Email, // Fax = c.Fax, // Phone = c.Phone // })) // .ForMember(vm => vm.TotalInvoices, map => map.MapFrom(c => c.Invoice.Count())); //Mapper.CreateMap<Employee, EmployeeViewModel>(); //Mapper.CreateMap<Genre, GenreViewModel>() // .ForMember(vm => vm.Tracks, map => map.MapFrom(g => g.Track.Select(t => t.TrackId).ToList())); //Mapper.CreateMap<InvoiceLine, InvoiceLineViewModel>(); //Mapper.CreateMap<MediaType, MediaTypeViewModel>() // .ForMember(vm => vm.Tracks, map => map.MapFrom(m => m.Track.Select(t => t.TrackId).ToList())); //Mapper.CreateMap<PlaylistTrack, PlaylistTrackViewModel>(); //Mapper.CreateMap<Playlist, PlaylistViewModel>(); } }
We left some lines commented out since we haven’t created yet the View Models. We are going to uncomment them one by one later on. At this point you should be able to resolve all dependencies inside the Startup class.
Shaping API responses
It’s high time to serve some data! We will start with the least complex example where we ‘ll try to serve customized responses for the Track entity. Let’s create the respective ViewModel first. Create the TrackViewModel class under a newly created folder named ViewModels at the root of the application.
public class TrackViewModel { public TrackViewModel() { } public int TrackId { get; set; } public int? AlbumId { get; set; } public int? Bytes { get; set; } public string Composer { get; set; } public int? GenreId { get; set; } public int MediaTypeId { get; set; } public int Milliseconds { get; set; } public string Name { get; set; } public decimal UnitPrice { get; set; } }
In case you didn’t notice, we have removed any navigation properties exist in the original entity Track cause this will be our simplest example. From now on, any time we add a ViewModel make sure you uncomment the respective line in the DomainToViewModelMappingProfile class. Before writing the first controller let’s add two important classes first. Create a new folder named Core under Infrastructure and paste the Expressions class.
public class Expressions { public static Expression<Func<Track, object>>[] LoadTrackNavigations() { Expression<Func<Track, object>>[] _navigations = { t => t.Album, t => t.Genre, t => t.InvoiceLine, t => t.MediaType }; return _navigations; } public static Expression<Func<Customer, object>>[] LoadCustomerNavigations() { Expression<Func<Customer, object>>[] _navigations = { c => c.Invoice, c => c.SupportRep }; return _navigations; } public static Expression<Func<Album, object>>[] LoadAlbumNavigations() { Expression<Func<Album, object>>[] _navigations = { a => a.Track, a => a.Artist }; return _navigations; } public static Expression<Func<Artist, object>>[] LoadArtistNavigations() { Expression<Func<Artist, object>>[] _navigations = { a => a.Album }; return _navigations; } }
This class has methods to inform Entity Framework what navigation properties to include when retrieving data from the Chinook database. The next class gave me serious headaches since is the one that contains the FilterProperties method which does all the magic work. The Utils.cs class is kind of large to paste here so just copy it from here. We will explain the FilterProperties behavior later on. Add a new folder named Controllers and create the first controller named TracksController.
[Route("api/[controller]")] public class TracksController : Controller { #region Properties private readonly ITrackRepository _trackRepository; private List<string> _properties = new List<string>(); private Expression<Func<Track, object>>[] includeProperties; private const int maxSize = 50; #endregion #region Constructor public TracksController(ITrackRepository trackRepository) { _trackRepository = trackRepository; _properties = new List<string>(); includeProperties = Expressions.LoadTrackNavigations(); } #endregion #region Actions public ActionResult Get(string props = null, int page = 1, int pageSize = maxSize) { try { var _tracks = _trackRepository.GetAll(includeProperties).Skip((page - 1) * pageSize).Take(pageSize); var _tracksVM = Mapper.Map<IEnumerable<Track>, IEnumerable<TrackViewModel>>(_tracks); string _serializedTracks = JsonConvert.SerializeObject(_tracksVM, Formatting.None, new JsonSerializerSettings() { ReferenceLoopHandling = ReferenceLoopHandling.Ignore }); JToken _jtoken = JToken.Parse(_serializedTracks); if (!string.IsNullOrEmpty(props)) Utils.FilterProperties(_jtoken, props.ToLower().Split(',').ToList()); return Ok(_jtoken); } catch (Exception ex) { return new HttpStatusCodeResult(500); } } [Route("{trackId}")] public ActionResult Get(int trackId, string props = null) { try { var _track = _trackRepository.Get(t => t.TrackId == trackId, includeProperties); if (_track == null) { return HttpNotFound(); } var _trackVM = Mapper.Map<Track, TrackViewModel>(_track); string _serializedTrack = JsonConvert.SerializeObject(_trackVM, Formatting.None, new JsonSerializerSettings() { ReferenceLoopHandling = ReferenceLoopHandling.Ignore }); JToken _jtoken = JToken.Parse(_serializedTrack); if (!string.IsNullOrEmpty(props)) Utils.FilterProperties(_jtoken, props.ToLower().Split(',').ToList()); return Ok(_jtoken); } catch (Exception ex) { return new HttpStatusCodeResult(500); } } #endregion }
OK, let’s explain what we actually do here. Firstly we make sure we include any navigation properties we want EF to include when retrieving Track entities. We have also set some pagination logic here with a maxsize = 50 items (in case isn’t explicit set on the request). After retrieving the data from the database we make the mapping with the respective View model. Later on we serialize the result and create a JToken representation. Finally and only if the user has set a respective set of properties to get using the props key on the query string, we pass this token on the FilterProperties method. The Get(int trackId, string props = null) action method works in the same way. Build and run the application to test it. Let’s retrieve all the tracks in their original form. This is the response when sending an /api/tracks request.
[ { "TrackId": 2, "AlbumId": 2, "Bytes": 5510424, "Composer": null, "GenreId": 1, "MediaTypeId": 2, "Milliseconds": 342562, "Name": "Balls to the Wall", "UnitPrice": 0.99 }, { "TrackId": 3, "AlbumId": 3, "Bytes": 3990994, "Composer": "F. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman", "GenreId": 1, "MediaTypeId": 2, "Milliseconds": 230619, "Name": "Fast As a Shark", "UnitPrice": 0.99 }, { "TrackId": 4, "AlbumId": 3, "Bytes": 4331779, "Composer": "F. Baltes, R.A. Smith-Diesel, S. Kaufman, U. Dirkscneider & W. Hoffman", "GenreId": 1, "MediaTypeId": 2, "Milliseconds": 252051, "Name": "Restless and Wild", "UnitPrice": 0.99 }, // code omitted
Now let’s assume your mobile clients needs to retrieve only the trackid, name and unitprice properties. All you have to do is send a request to api/tracks?props=trackid,name,unitprice.
[ { "TrackId": 2, "Name": "Balls to the Wall", "UnitPrice": 0.99 }, { "TrackId": 3, "Name": "Fast As a Shark", "UnitPrice": 0.99 }, { "TrackId": 4, "Name": "Restless and Wild", "UnitPrice": 0.99 }, { "TrackId": 5, "Name": "Princess of the Dawn", "UnitPrice": 0.99 }, // code omitted
The FilterProperties method removed all other properties you didn’t mention in the props key. At this point we can understand that if we want to get a certain set of properties we only need to defined them as a comma separated string value on the props query string key. If we don’t define the props key then all the properties will be returned.
Navigations
Things are getting serious when we deal with entities that contains Navigation properties. We ‘ll start with the AlbumViewModel which contains a collection of TrackViewModel items. Add it in the ViewModels folder and of course uncomment the respective line in the AutoMapperConfiguration class.
public class AlbumViewModel { public AlbumViewModel() { Track = new HashSet<TrackViewModel>(); } public int AlbumId { get; set; } public string ArtistName { get; set; } public string Title { get; set; } public virtual ICollection<TrackViewModel> Track { get; set; } }
And of course the AlbumsController.
[Route("api/[controller]")] public class AlbumsController : Controller { #region Properties private readonly IAlbumRepository _albumRepository; private List<string> _properties = new List<string>(); private Expression<Func<Album, object>>[] includeProperties; private const int maxSize = 50; #endregion #region Constructor public AlbumsController(IAlbumRepository albumRepository) { _albumRepository = albumRepository; _properties = new List<string>(); includeProperties = Expressions.LoadAlbumNavigations(); } #endregion #region Actions public ActionResult Get(string props = null, int page = 1, int pageSize = maxSize) { try { var _albums = _albumRepository.GetAll(includeProperties).Skip((page - 1) * pageSize).Take(pageSize); var _albumsVM = Mapper.Map<IEnumerable<Album>, IEnumerable<AlbumViewModel>>(_albums); string _serializedAlbums = JsonConvert.SerializeObject(_albumsVM, Formatting.None, new JsonSerializerSettings() { ReferenceLoopHandling = ReferenceLoopHandling.Ignore }); JToken _jtoken = JToken.Parse(_serializedAlbums); if (!string.IsNullOrEmpty(props)) Utils.FilterProperties(_jtoken, props.ToLower().Split(',').ToList()); return Ok(_jtoken); } catch (Exception ex) { return new HttpStatusCodeResult(500); } } [Route("{albumId}")] public ActionResult Get(int albumId, string props = null) { try { var _album = _albumRepository.Get(t => t.AlbumId == albumId, includeProperties); if (_album == null) { return HttpNotFound(); } var _albumVM = Mapper.Map<Album, AlbumViewModel>(_album); string _serializedAlbum = JsonConvert.SerializeObject(_albumVM, Formatting.None, new JsonSerializerSettings() { ReferenceLoopHandling = ReferenceLoopHandling.Ignore }); JToken _jtoken = JToken.Parse(_serializedAlbum); if (!string.IsNullOrEmpty(props)) Utils.FilterProperties(_jtoken, props.ToLower().Split(',').ToList()); return Ok(_jtoken); } catch (Exception ex) { return new HttpStatusCodeResult(500); } } #endregion }
Let’s start by making a request to /api/albums (the first 50 albums) without defining any properties.
[ { "AlbumId": 2, "ArtistName": "Accept", "Title": "Balls to the Wall", "Track": [ { "TrackId": 2, "AlbumId": 2, "Bytes": 5510424, "Composer": null, "GenreId": 1, "MediaTypeId": 2, "Milliseconds": 342562, "Name": "Balls to the Wall", "UnitPrice": 0.99 } ] }, { "AlbumId": 3, "ArtistName": "Accept", "Title": "Restless and Wild", "Track": [ { "TrackId": 3, "AlbumId": 3, "Bytes": 3990994, "Composer": "F. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman", "GenreId": 1, "MediaTypeId": 2, "Milliseconds": 230619, "Name": "Fast As a Shark", "UnitPrice": 0.99 }, { "TrackId": 4, "AlbumId": 3, "Bytes": 4331779, "Composer": "F. Baltes, R.A. Smith-Diesel, S. Kaufman, U. Dirkscneider & W. Hoffman", "GenreId": 1, "MediaTypeId": 2, "Milliseconds": 252051, "Name": "Restless and Wild", "UnitPrice": 0.99 }, // code omitted
Track property is a collection of TrackViewModel items. Let’s focus on the 3rd album and request only the ArtistName, Title and Track properties requesting /api/albums/3?props=artistname,title,track.
{ "ArtistName": "Accept", "Title": "Restless and Wild", "Track": [ { "TrackId": 3, "AlbumId": 3, "Bytes": 3990994, "Composer": "F. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman", "GenreId": 1, "MediaTypeId": 2, "Milliseconds": 230619, "Name": "Fast As a Shark", "UnitPrice": 0.99 }, { "TrackId": 4, "AlbumId": 3, "Bytes": 4331779, "Composer": "F. Baltes, R.A. Smith-Diesel, S. Kaufman, U. Dirkscneider & W. Hoffman", "GenreId": 1, "MediaTypeId": 2, "Milliseconds": 252051, "Name": "Restless and Wild", "UnitPrice": 0.99 }, // code omitted } ] }
As we can see the nested TrackViewModel items inside the Track collection were returned with all their properties. What we would like to have though is the ability to cut certain properties for those items as well. So how can we do it? Simply, by encapsulating the nested properties, semicolon separated inside parenthesis. Assuming we only want the trackid and unitprice for each track we would send the request /api/albums/3?props=artistname,title,track(trackid;unitprice).
{ "ArtistName": "Accept", "Title": "Restless and Wild", "Track": [ { "TrackId": 3, "UnitPrice": 0.99 }, { "TrackId": 4, "UnitPrice": 0.99 }, { "TrackId": 5, "UnitPrice": 0.99 } ] }
Let’s keep the pace up and procceed with the ArtistViewModel which has a collection of AlbumViewModel items.
public class ArtistViewModel { public ArtistViewModel() { Album = new HashSet<AlbumViewModel>(); } public int ArtistId { get; set; } public string Name { get; set; } public virtual ICollection<AlbumViewModel> Album { get; set; } }
[Route("api/[controller]")] public class ArtistsController : Controller { #region Properties private readonly IArtistRepository _artistRepository; private List<string> _properties = new List<string>(); private const int maxSize = 50; #endregion #region Constructor public ArtistsController(IArtistRepository artistRepository) { _artistRepository = artistRepository; _properties = new List<string>(); } #endregion #region Actions public ActionResult Get(string props = null, int page = 1, int pageSize = maxSize) { try { var _artists = _artistRepository.LoadAll().Skip((page - 1) * pageSize).Take(pageSize); var _artistsVM = Mapper.Map<IEnumerable<Artist>, IEnumerable<ArtistViewModel>>(_artists); string _serializedArtists = JsonConvert.SerializeObject(_artistsVM, Formatting.None, new JsonSerializerSettings() { ReferenceLoopHandling = ReferenceLoopHandling.Ignore }); JToken _jtoken = JToken.Parse(_serializedArtists); if (!string.IsNullOrEmpty(props)) Utils.FilterProperties(_jtoken, props.ToLower().Split(',').ToList()); return Ok(_jtoken); } catch (Exception ex) { return new HttpStatusCodeResult(500); } } [Route("{artistId}")] public ActionResult Get(int artistId, string props = null) { try { var _artist = _artistRepository.Load(artistId); if (_artist == null) { return HttpNotFound(); } var _artistVM = Mapper.Map<Artist, ArtistViewModel>(_artist); string _serializedArtist = JsonConvert.SerializeObject(_artistVM, Formatting.None, new JsonSerializerSettings() { ReferenceLoopHandling = ReferenceLoopHandling.Ignore }); JToken _jtoken = JToken.Parse(_serializedArtist); if (!string.IsNullOrEmpty(props)) Utils.FilterProperties(_jtoken, props.ToLower().Split(',').ToList()); return Ok(_jtoken); } catch (Exception ex) { return new HttpStatusCodeResult(500); } } #endregion }
Let’s focus on a specific Artist making a request to /api/artists/6.
{ "ArtistId": 6, "Name": "Antônio Carlos Jobim", "Album": [ { "AlbumId": 8, "ArtistName": "Antônio Carlos Jobim", "Title": "Warner 25 Anos", "Track": [ { "TrackId": 63, "AlbumId": 8, "Bytes": 5990473, "Composer": null, "GenreId": 2, "MediaTypeId": 1, "Milliseconds": 185338, "Name": "Desafinado", "UnitPrice": 0.99 }, { "TrackId": 64, "AlbumId": 8, "Bytes": 9348428, "Composer": null, "GenreId": 2, "MediaTypeId": 1, "Milliseconds": 285048, "Name": "Garota De Ipanema", "UnitPrice": 0.99 }, // code omitted ] }, { "AlbumId": 34, "ArtistName": "Antônio Carlos Jobim", "Title": "Chill: Brazil (Disc 2)", "Track": [ { "TrackId": 391, "AlbumId": 34, "Bytes": 9141343, "Composer": "Vários", "GenreId": 7, "MediaTypeId": 1, "Milliseconds": 279536, "Name": "Garota De Ipanema", "UnitPrice": 0.99 }, { "TrackId": 392, "AlbumId": 34, "Bytes": 7143328, "Composer": "Vários", "GenreId": 7, "MediaTypeId": 1, "Milliseconds": 213237, "Name": "Tim Tim Por Tim Tim", "UnitPrice": 0.99 }, //code omitted ] } ] }
This artist has two albums with each album containing a set of tracks. Assuming a view in your mobile application needs to display only the names of those albums you would make a request to /api/artists/6?props=album(title).
{ "Album": [ { "Title": "Warner 25 Anos" }, { "Title": "Chill: Brazil (Disc 2)" } ] }
In case you also wanted to display the trackids contained in each album and their unitprice you would make a request to /api/artists/6?props=album(title;track(trackid;unitprice)).
{ "Name": "Antônio Carlos Jobim", "Album": [ { "Title": "Warner 25 Anos", "Track": [ { "TrackId": 63, "UnitPrice": 0.99 }, { "TrackId": 64, "UnitPrice": 0.99 }, // code omitted ] }, { "Title": "Chill: Brazil (Disc 2)", "Track": [ { "TrackId": 391, "UnitPrice": 0.99 }, { "TrackId": 392, "UnitPrice": 0.99 }, // code omitted ] } ] }
I left the api/customers resource last cause it’s the most complex. First add the following view models.
public class InvoiceViewModel { public InvoiceViewModel() { InvoiceLine = new HashSet<InvoiceLineViewModel>(); } public int InvoiceId { get; set; } public string BillingAddress { get; set; } public string BillingCity { get; set; } public string BillingCountry { get; set; } public string BillingPostalCode { get; set; } public string BillingState { get; set; } public int CustomerId { get; set; } public DateTime InvoiceDate { get; set; } public decimal Total { get; set; } public ICollection<InvoiceLineViewModel> InvoiceLine { get; set; } }
public class InvoiceLineViewModel { public int InvoiceLineId { get; set; } public int InvoiceId { get; set; } public int Quantity { get; set; } public int TrackId { get; set; } public decimal UnitPrice { get; set; } }
public class AddressViewModel { public string Address { get; set; } public string City { get; set; } public string Country { get; set; } public string PostalCode { get; set; } public string State { get; set; } }
public class ContactViewModel { public string Email { get; set; } public string Fax { get; set; } public string Phone { get; set; } }
public class CustomerViewModel { public CustomerViewModel() { } public int CustomerId { get; set; } public string Company { get; set; } public string FirstName { get; set; } public string LastName { get; set; } public int? SupportRepId { get; set; } public int TotalInvoices { get; set; } public ICollection<InvoiceViewModel> Invoice { get; set; } public AddressViewModel Address { get; set; } public ContactViewModel Contact { get; set; } }
Make sure you uncomment their AutoMapper configurations as well. A customer may have many invoices which in turn may contain many invoiceline items. I have also created a new ViewModel to save all the address related information in a navigation property named Address and another one for Contact information. Here’s the controller.
[Route("api/[controller]")] public class CustomersController : Controller { #region Properties private readonly ICustomerRepository _customerRepository; private List<string> _properties = new List<string>(); private const int maxSize = 50; #endregion #region Constructor public CustomersController(ICustomerRepository customerRepository) { _customerRepository = customerRepository; _properties = new List<string>(); } #endregion #region Actions public ActionResult Get(string props = null, int page = 1, int pageSize = maxSize) { try { var _customers = _customerRepository.LoadAll().Skip((page - 1) * pageSize).Take(pageSize); var _customersVM = Mapper.Map<IEnumerable<Customer>, IEnumerable<CustomerViewModel>>(_customers); string _serializedCustomers = JsonConvert.SerializeObject(_customersVM, Formatting.None, new JsonSerializerSettings() { ReferenceLoopHandling = ReferenceLoopHandling.Ignore }); JToken _jtoken = JToken.Parse(_serializedCustomers); if (!string.IsNullOrEmpty(props)) Utils.FilterProperties(_jtoken, props.ToLower().Split(',').ToList()); return Ok(_jtoken); } catch (Exception ex) { return new HttpStatusCodeResult(500); } } [Route("{customerId}")] public ActionResult Get(int customerId, string props = null) { try { var _customer = _customerRepository.Load(customerId); if (_customer == null) { return HttpNotFound(); } var _customerVM = Mapper.Map<Customer, CustomerViewModel>(_customer); string _serializedCustomer = JsonConvert.SerializeObject(_customerVM, Formatting.None, new JsonSerializerSettings() { ReferenceLoopHandling = ReferenceLoopHandling.Ignore }); JToken _jtoken = JToken.Parse(_serializedCustomer); if (!string.IsNullOrEmpty(props)) Utils.FilterProperties(_jtoken, props.ToLower().Split(',').ToList()); return Ok(_jtoken); } catch (Exception ex) { return new HttpStatusCodeResult(500); } } #endregion }
Let’s start with a request to /api/customers/5 that serves all the available data.
{ "CustomerId": 5, "Company": "JetBrains s.r.o.", "FirstName": "František", "LastName": "Wichterlová", "SupportRepId": 4, "TotalInvoices": 7, "Invoice": [ { "InvoiceId": 77, "BillingAddress": "Klanova 9/506", "BillingCity": "Prague", "BillingCountry": "Czech Republic", "BillingPostalCode": "14700", "BillingState": null, "CustomerId": 5, "InvoiceDate": "2009-12-08T00:00:00", "Total": 1.98, "InvoiceLine": [ { "InvoiceLineId": 417, "InvoiceId": 77, "Quantity": 1, "TrackId": 2551, "UnitPrice": 0.99 }, { "InvoiceLineId": 418, "InvoiceId": 77, "Quantity": 1, "TrackId": 2552, "UnitPrice": 0.99 } ] }, { "InvoiceId": 100, "BillingAddress": "Klanova 9/506", "BillingCity": "Prague", "BillingCountry": "Czech Republic", "BillingPostalCode": "14700", "BillingState": null, "CustomerId": 5, "InvoiceDate": "2010-03-12T00:00:00", "Total": 3.96, "InvoiceLine": [ { "InvoiceLineId": 535, "InvoiceId": 100, "Quantity": 1, "TrackId": 3254, "UnitPrice": 0.99 }, // code omitted ] }, { "InvoiceId": 122, "BillingAddress": "Klanova 9/506", "BillingCity": "Prague", "BillingCountry": "Czech Republic", "BillingPostalCode": "14700", "BillingState": null, "CustomerId": 5, "InvoiceDate": "2010-06-14T00:00:00", "Total": 5.94, "InvoiceLine": [ { "InvoiceLineId": 653, "InvoiceId": 122, "Quantity": 1, "TrackId": 457, "UnitPrice": 0.99 }, // code omitted ] }, // code omitted ], "Address": { "Address": "Klanova 9/506", "City": "Prague", "Country": "Czech Republic", "PostalCode": "14700", "State": null }, "Contact": { "Email": "frantisekw@jetbrains.com", "Fax": "+420 2 4172 5555", "Phone": "+420 2 4172 5555" } }
Now let’s request only two properties for each navigation property from top to bottom by making a request to /api/customers/5?props=company,invoice(total;invoiceline(invoiceid;quantity)),address(address;city),contact(email;fax)..
{ "Company": "JetBrains s.r.o.", "Invoice": [ { "Total": 1.98, "InvoiceLine": [ { "InvoiceId": 77, "Quantity": 1 }, { "InvoiceId": 77, "Quantity": 1 } ] }, { "Total": 3.96, "InvoiceLine": [ { "InvoiceId": 100, "Quantity": 1 }, // code omitted ] }, { "Total": 16.86, "InvoiceLine": [ { "InvoiceId": 306, "Quantity": 1 }, // code omitted ] } // code omitted ], "Address": { "Address": "Klanova 9/506", "City": "Prague" }, "Contact": { "Email": "frantisekw@jetbrains.com", "Fax": "+420 2 4172 5555" } }
Discussion
Let’s talk Performance. Can FilterProperties method increase the time the server processes the request? The answer depends on the amount of data you request. In case you remove the pagination logic and make a request to /api/tracks?props=bytes,composer,milliseconds the server will response in about 30 seconds which is not acceptable. The reason behind this is that the algorithm will try to process 3500 JToken items one by one and will end up to remove 21000 nested JTokens.
One the other hand a request to /api/tracks?props=bytes,composer,milliseconds&pagesize=500 will only last 1-2 seconds maximum as it would take in normal circumstances. So keep in mind to always use pagination with this technique. It’s important to apply the pagination before you pass the original JToken to the FilterProperties method. Since we are talking about perfonmance I would like here to make a proposal. The algorithm and the FilterProperties method can be optimized and support many other features as well. I have tested it against the Chinook database and some others and it seemed to work fine but it can be done even better. If anyone wants to contribute and improve the algorithm be my guest. In order to help you with this I have created some Views in the application which display the responses for various API calls with a pretty and scrollable enabled JSON format. You can also click the link on each panel header to open the request on a separate tab.
The API calls are made through AngularJS controllers. Let’s see part of the tracksCtrl controller.
(function (app) { 'use strict'; app.controller('tracksCtrl', tracksCtrl); tracksCtrl.$inject = ['$scope', '$http']; function tracksCtrl($scope, $http) { // api/tracks $http.get('api/tracks/'). success(function (data, status, headers, config) { $scope.apiTracks = data; }). error(function (data, status, headers, config) { console.log(data); }); // api/tracks?page=2&pagesize=100 $http.get('api/tracks?page=2&pagesize=100'). success(function (data, status, headers, config) { $scope.apiTracksPaged = data; }). error(function (data, status, headers, config) { console.log(data); }); // code omitted
The data are bound to the view through some angularJS directives as follow:
<div class="col-md-6"> <div class="panel panel-primary"> <div class="panel-heading"> <h3 class="panel-title"><a href="api/tracks" target="_blank">api/tracks</a></h3> </div> <div class="panel-body"> <img src="~/images/spinner.gif" class="spinner" ng-show="!apiTracks" /> <perfect-scrollbar ng-show="apiTracks" class="api-box" wheel-propagation="true" wheel-speed="10" min-scrollbar-length="20"> <pretty-json json='apiTracks' replace-key='replaceKey' replace-value='replaceValue' indent-key='4' indent-value='indentValue'></pretty-json> </perfect-scrollbar> </div> </div> </div> <div class="col-md-6"> <div class="panel panel-primary"> <div class="panel-heading"> <h3 class="panel-title"><a href="api/tracks?page=2&pagesize=100" target="_blank">api/tracks?page=2&pagesize=100</a></h3> </div> <div class="panel-body"> <img src="~/images/spinner.gif" class="spinner" ng-show="!apiTracksPaged" /> <perfect-scrollbar ng-show="apiTracksPaged" class="api-box" wheel-propagation="true" wheel-speed="10" min-scrollbar-length="20"> <pretty-json json='apiTracksPaged' replace-key='replaceKey' replace-value='replaceValue' indent-key='4' indent-value='indentValue'></pretty-json> </perfect-scrollbar> </div> </div> </div>
So in case you want to improve the algorithm and the function (naming, regression, performance etc..) make sure you pass all those tests first and then send me a pull request to examine. Hint: In production if the algorithm crashes Try/Catch block make sure to return the original JToken just in case. In development leave the function as it is in order to understand what went wrong and solve the problem. When you download the project from GitHub make sure to run the following command to download Bower dependencies.
bower install
Does it worth it? Absolutely. Decreasing the amount of data passing through the wire and reducing CPU and memory usage on the clients is the #1 TOP Priority when designing APIs. We managed to keep a single API giving the ability to the clients to request only the parts of the data that really needs to process. Maintainability and testability remained at the same levels as if you were designing the API for a single client.
That’s it we have finally finished! I hope you have enjoyed the post as much as I did creating it. You can download the project we built from my GitHub account as always. The project was built on Visual Studio 2015 and ASP.NET Core but the parts that do all the magic work can be copy-pasted in any other versions you wish.
In case you find my blog’s content interesting, register your email to receive notifications of new posts and follow chsakell’s Blog on its Facebook or Twitter accounts.
.NET Web Application Development by Chris S. | |||