Data Structure Organization for Web Services

Many of the PWS web services fall into the category of CRUD (Create, Read, Update, Delete) services. That is, they are concerned with adding, editing or deleting data in Projector. A typical set of CRUD services includes:

• A listing service to retrieve a listing of instances of the given entity. A small number of key attributes is included in the response for each instance.
• A retrieval service to retrieve a single instance or a small number of specific instances of the given entity. A complete set of attributes for each retrieved instance is included in the response.
• A save service to save changes to existing instances or create new instances of the given entity.
• A delete service to delete an existing instance of the given entity.

In some cases, where an entity is subordinate to another entity (for example, roles, being subordinate to projects), instead of a separate listing service and retrieval service, the retrieval service for the enclosing entity includes a collection of instances of the subordinate entity (For example, the response from the retrieval service for projects includes the collection of roles on the project).

The data structures employed by these services typically follow a certain pattern. This pattern consists of the following general types of data structures:

• Reference Structures
• Summary Structures
• Detail Structures
• Element Structures
• Summary Element Structures

Each structure has a specific purpose or specific purposes, and generally adheres to certain rules and conventions as detailed below.

Reference Structures

Reference Structures uniquely identify a specific instance of an entity (for example, a project reference structure identifies a specific project). Where specific instances of an entity may be uniquely identified by multiple different keys, the reference structure will provide each alternative means of identification (For example, projects may be identified by an internal identifier, a unique identifier or the project code – the project reference structure offers all three means of identification).

For a reference structure to be considered non-empty, at least one means of identification must be specified. It is acceptable to specify multiple means of identification, however the specified key values must all refer to the same instance in order for the reference to be considered valid.

Reference structures are used standalone to identify instances, as in the case of services that retrieve or delete an instance of an entity and must take as input a parameter that specifies which instance is to be retrieved or deleted. Reference structures are also used as the base class for summary and detail structures. In this way, the identifying attributes of an entity become part of the data structures that represent them more fully.

Note on internal identifiers: Most reference structures contain an internal identifier as one of the alternative means for identifying instances. Internal identifiers are mutable, and while not likely to happen frequently, they are subject to changing over time. As such, they are typically reserved for internal use and by default, the values returned by web services for internal identifiers will be NULL and any internal identifiers specified as inputs to web services will be ignored. Unique identifiers, by contrast, are immutable and should be used by web service consumers instead of internal identifiers.

Summary Structures

Summary structures inherit from reference structures and are typically used in services that return lists of entities. They include a small number of key attributes that are generally considered useful in lists. The size of the structure is intentionally kept pretty small to reduce the size of web service responses containing lists of these structures.

Detail Structures

Detail structures inherit from summary structures and are intended to contain (directly or via inheritance) the complete set of attributes describing the underlying entity. These structures are used as input to services that save changes to instances of an entity, and are part of the response from services that retrieve instances. This makes it easy to use the web services to make changes to an instance, as the same structure retrieved by one service call may be edited and passed back up to a different service to save the changes.

In some cases, a detail structure may contain collections of entities. These entities are subordinate to the main entry, but are managed by the web service as if they were collectively a single attribute of the main entity. That is, the entire list may be left as is, replaced wholesale or cleared, but individual items may not be added, modified or removed.

Element Structures

Element structures contain detail structures, and additionally may contain attributes that are related to the entity and are useful to include with the data about the entity itself, but are not directly attributes of the entity (for example, permission flags). Services that retrieve an instance of a particular entity typically contain an element structure in the response.

In some cases, an element structure may contain a collection of subordinate element structures. In these cases, there are separate web services for retrieving, saving and deleting  instances of the subordinate entry.

The aim of element structures is to allow the content of detail structures to be restricted only to direct attributes of the entity, while allowing for the inclusion of additional, related and useful attributes when retrieving instances of a given entity. Organizing the data in this manner means that the detail structure retrieved by a retrieval web service can be passed back up to the corresponding save web service, while avoiding sending extraneous information up to the service.

Summary Element Structures

When a listing of instances of a certain entity needs to contain attributes that are not direct attributes of the entity, summary element structures are used. In the same way that element structures allow detail structures to be limited to direct attributes of the underlying entity, summary element structures allow summary structures to be similarly restricted.

On updating data using web services:

There are two approaches to using web services that save changes to a particular entity:

• Full image. In this approach, one would use the retrieval web service to retrieve the element structure for a specific instance of the entity, then make necessary modifications to the detail structure contained within the element structure, and pass the entire detail structure as input to the corresponding save web service.
• Targeted attribute. In this approach, one would specify only the attributes that are to be changed, and specify that other attributes are to be left as is. Typically this involves constructing a detail structure from scratch, populating the fields that are to be changed and leaving the rest NULL.

The first approach is fairly straightforward, however there are a couple details to cover on the second approach:

Nullable vs Non-nullable Attributes

Generally, the save services are implemented such that leaving an attribute NULL in the detail structure is an instruction to leave the current value of the attribute unchanged. This works file for non-nullable attributes, as NULL is not a valid value for the attribute itself, but this is not the case for nullable attributes. For nullable attributes, there needs to be a way to set the value, NULL the value out or leave it alone. Nullable attributes have a corresponding clear flag that facilitates this as follows:

Grouped Attributes

In some cases, an entity may contain a group of two or more attributes that are related in such a way that it makes more sense to treat the group as a unit rather than independently. In these cases, where a clear flag is appropriate, it will typically apply to the group as a whole.