It is common to populate dropdowns, grids and general arrays from backend REST APIs. Here is a simple reusable chained component which supports instrumentation, schema validation, abstraction and polymorphism. The data is populated by chaining as follows:
Using the Array Data Source described in this post
// populating an array
var dataSource = [];
dataSource.ajax('domain.com/path/get?a=1&b=2');
// populating a KnockoutJS observable array
var observableArray = ko.ObservableArray();
bservableArray.ajax('domain.com/path/get?a=1&b=2' );
// adding all optional parameters, including validation and completion and record transformation callbacks
var schema = { field1:'string', field2:'date', ... };
observableArray.ajax('domain.com/path/get?a=1&b=2'
, schema
, function() { /* perform logic with full data set */ },
, function() { /* perform transformation of each record */ } );
The high level sequence diagram is depicted below. The entry point is upon invocation of the main data source function by the user (e.g., above examples). After accounting for input variability, reporting the request to the logger and saving a reference to the (input) array to be populated, a jQuery ajax method is being invoked. The server-side REST API implementation receives the requests and performs the operations necessary to retrieve the data; a corresponding event is reported to the logger to enable paring the server-side event with the client side event. Upon successful response, the data is parsed and validated; report any errors to the logger. Subsequently, the data is populated in the array and a 3rd event is reported to the logger, allowing it to group all three events into a single package. Finally, the completion callback is invoked.
The code implementing the above sequence is provided below. Note the following key patterns implemented:
Polymorphism: We are implementing the ajax() method across both the base Array and KnockoutJS ObservableArray() objects. To be able to clear the array prior to populating it, we need to define the removeAll() method, commonly used with ObservableArray(), for the base Array object.
Defining by Array.prototype.removeAll is not safe because it defines the new function as enumerable and injects a new (function) object into the loop for (var p in this) {}; this is one reason not to use a generic property iterator when the intention is to iterate over the array’s elements. Instead of direct assignment to the Array.prototype, we need to use Object.defineProperty to define the new method as a non-enumerable property of Array.prototype.
The same exact pattern appears below when defining the ajax() method for the Array prototype; this is the route taken when invoking dataSource.ajax in case the dataSource is a base Array. The KnockoutJS extension is defined by assigning the function to ko.observableArray.fn.ajax; this is safe because KnockoutJS does not support iterating over the array directly but rather on the return of the observableArray() method, which will avoid adding any non-data enumerables.
For older browsers which do not implement Object.defineProperty we still need to perform the assignment directly. Careful however, because such direct assignment will break all legacy code performing general for (p in array) loops as it will iterate over the newly added function elements as well.
Overloading: The parameters of schema, completeCallback and recordCallback are optional. To handle situations where they are missing, we add code shuffling values as needed. Specifically, if the schema parameter is actually a function, we shift the input variables assignments and assign it to null; this ensure correct behavior downstream within the function.
Validation: The schema is defined as a standard JSON structure, where the values of each property is the type it is to be validated against. There are required and optional types; each type has a specific format associated with it. See the separate post about schema validation about validating the URL request. Note that in this example, validation is performed on the REST API response; not on the input ‘request’ (which is a URL with a query string).
Inline Transformation: The data received from the backend REST API may be transformed on a record-by-record basis, so as to enable adapting to pre-existing UI. This is enabled using the recordCallback(), which is invoked once for each record retrieved.
Instrumentation: Instrumentation is structured logging supporting subsequent analysis. The data source function automatically sends events to a logger; it is expected that the REST API back-end sends log events too. If this pattern is followed, the log should show three (3) events for each request: the initial request (prior to invocation of jQuery ajax), the processing of the REST API, and the subsequent parsing and validation by the jQuery ajax success() callback; some may argue that the REST API should log two events, one for the request entry, and one for completion, to support profiling. Subsequent analysis of the logs will allow identifying situations where log-groups are incomplete, e.g., in case the REST API did not receive a client request, or in case the client received no response or an invalid response. See other posts for details on instrumentation patterns .
Clearing Arrays: Javascript does not have a standardized method for clearing arrays; assigning to an empty array, e.g., array=[], creates a new object, breaks all existing references, and often results in code breakage. This post shows a simple and effective solution for creating a removeAll() method without impacting existing references and array iterators.
Polymorphic Ajax Data Source
var removeAll_prototype = function () { while (this.length > 0) this.pop(); };
if (Object.defineProperty)
Object.defineProperty(Array.prototype, 'removeAll', {
enumerable:false, // this is the default value
value: removeAll_prototype // this is the function defined above
});
else
Array.prototype.removeAll = removeAll_prototype
;
var ajax_prototype = function (url, schema, completeCallback, recordCallback) {
// The schema parameters and the callbacks are optional
if (typeof schema === 'function') {
recordCallback=completeCallback; completeCallback=schema; schema=null;
}
// for details, see a seperate post on instrumentation
log.reportRequest('datasource',url)
// Save a reference to the (input) array for use in the ajax success callback
var array = this;
$.ajax({
type: 'GET', url: url, cache: false
, success: function (json) {
try {
// Parse JSON response
var data = $.parseJSON(json.JSONencode());
// Validate (see other posts for schema and validation patterns)
if (schema) data.validate(schema);
// Prior to populating, clear the data
if (array.removeAll) array.removeAll();
// Populate observable
for (var i = 0; i < data.length; i++) {
if (recordCallback) recordCallback(data[i]);
array.push(data[i]);
}
} // for error reporting patterns see other posts
catch (e) { log.reportError(e.message); }
}
, error: function (msg) { log.reportError(msg.statusText); }
, complete: function () { if (completeCallback) completeCallback(); }
});
}
ko.observableArray.fn.ajax = ajax_prototype;
if (Object.defineProperty)
Object.defineProperty(Array.prototype, 'ajax', {
enumerable: false, // this is the default value
value: ajax_prototype
});
else
Array.prototype.ajax = ajax_prototype;
This concept can be extended for binding any client-side component to the back-end data, offering full polymorphism, validation and instrumentation, and without the need to explicitly write ajax invocations and bindings.
Cloud OS Patterns 
Leave a comment