Introduction to the Ivory Programming System

Ivory is a general-purpose computer programming system. It is intended for applications modelled with an emphasis on data rather than process. This is sometimes referred to as declarative, as the aim is to concentrate more on entities and the relationships between them rather than the computational steps needed to generate outputs and state changes.

What are its key features?

Uniform presentation of data

What this means is that the data types are consistent and uniform throughout the system. This particularly applies to data structures.

Native support for dynamic data structures and types

This means providing the tools to implement flexible data models, such as those based on general property sets and arbitrary values, which has become widely accepted.

Higher order data types, including unevaluated expressions

Function types and unevaluated expressions have many uses. Primarily they improve referential transparency and are very useful for call-backs and iterators.

Independent Data Stores

Data stores in an Ivory system may contain values of the full range of data types, including types local only to the store. For an application involving multiple stores, each is treated independently by the run-time system in a way that is transparent to the programming level.

Data representation and processing efficiency

For large volumes of data, representation and processing efficiency is clearly important. Taking the type for a basic property set as an illustration, its representation is a vector (array) of Name/Expression pairs. For many properties, this means in practice that each requires about 8 bytes of storage.

For small property sets, access to individual named properties involves a loop. This currently takes just 5 instructions on a PC architecture.

Adoption of functional programming technologies

The Ivory system inherits much from functional programming, both in the way that data is structured, and the underlying run-time system (essentially a derivative of an STG machine).

Component based modelling

In an analogous way to the integrated circuits in an electronic system, an Ivory application is expected to rely heavily on existing black-box built-in component methods (functions) at a fine level. The point here is that the application components and relationships are in a data store and not embedded in code. This means that it is possible for an organization to build a common growing repository of core functionality, with different applications determined solely by their data store configurations.

Rule based scalability

Data change sensitive rules eliminate the need for almost all direct forward function calls for application state changes. This enables applications to scale with less risk of breaking existing functionality. Built-in primitives are rarely changed, just new ones added. For applications that are non-stop, it is possible (with care) to change the behaviour without the need for a re-start.

Native code execution via a C++ code generator

There is a compiler for the system programming language Ivoryscript that outputs C++ source code so that built-in functions can run at the speed of native code. For new functionality, a prototype can often first be constructed in a data store and later built in as it matures - with little or no modification.

Interpreted code execution via a byte code generator

The system programming language Ivoryscript is used to configure data stores via a byte code interpreter. With a suitable user interface, it is also used to query and change application data.


Although in an experimental phase, some of the applications that an Ivory system has been used for are:

1.Overview

The system is manifested as a hybrid of a number of existing technologies, but the emphasis is on permitting late binding of names to values; thus it is intended for applications where flexibility and dynamic re-configuration is the primary interest, rather than pure execution speed.

It comprises a number of data stores and a means to evaluate expressions. Whilst it is possible to obtain a result by direct evaluation, the intended computational model is summarised by Figure 1. An application's state is maintained in one or more stores, with events triggering threads of computations called sparks.



Figure 1: Ivory Model


A spark's execution involves evaluating an expression that typically refers to stored data. The model is very dynamic in that sparks may copy or update data for subsequent sparks.

Expressions are defined according to the syntax of a language called IvoryScript, which is essentially a simplified Haskell-like lazy functional language. It includes higher order function types, type classes, non-strict semantics and a mix of static and dynamic type checking. There are also reference and name types intended for dynamic structures. Evaluation is performed either by built-in native or interpreted code; there is a compiler which generates either C++ or byte code for the Ivory system virtual machine.

The Ivory memory model is as follows: The evaluation of an expression uses its own stack and heap space and the only data that can be shared across distinct evaluations must have been allocated in the store. When an evaluation completes its execution, its heap and stack spaces are reclaimed. In order to ensure that no dangling pointer is created in the store by such a memory collection, all updates in the store entail a deep copy of data from the spark memory space to the store.

Even though IvoryScript is a Turing complete programming language, sparks are intended to be short lived. Based on theses premises, the current implementation of the run-time system does not use a garbage collector; all the memory space used by a spark is reclaimed at the end of its execution.

2.Data Stores

The Ivory system provides integrated data storage. This is based around the notion of a set of objects associated with what is termed an active data store (ADS). The term active arises from expressions with side effects, which are able to modify both the expressions within a store or to interact with the external environment.

Objects are evaluated expressions referred to by values of type Ref and subtypes of Ref distinguish objects; furthermore, the type of every object can be determined at run-time.

Ivory is not object-oriented in that objects are primarily storage entities and are not directly related to type classes. However because the type of an object can be obtained at run-time, type class methods may be dispatched dynamically to achieve a similar result.

Special objects enable stores to provide a localised global namespace. This is subdivided into a tree hierarchy using a conventional naming scheme modelled closely on the files and directories within a network filing system.

There are three different types of store with a transparent access mechanism. These are:

  1. Local
      Data is maintained in main memory and exists for the lifetime of an Ivory session;
  2. Persistent
      Data is maintained in secondary storage and persists between Ivory sessions;
  3. Remote
      Data is maintained by a remote process.

Given the inevitable additional overheads imposed by such a scheme, it is considered important that an Ivory system implements data stores with reasonable efficiency. The aim is support access rates orders of magnitude faster than could be achieved with ,say, a relational database.

The Ref type means that there may be more than one reference to the same data. For this reason reverse references are maintained. They simultaneously provide reference counts as well as a useful navigation capability for component-based application models.

Sets of store names are implemented as namespace objects and each store has a special one known as its root namespace. Access is via the Select class operator `.': Thus: .x refers to a named value in the root namespace of the root store.

Values are uniquely identified by a pathname resulting from a hierarchy based on separate namespaces provided implicitly by ADS or explicitly by Namespace objects. An example of a pathname is:

.x.y.z

This would refer to a local name z to be found in the explicit or implicit namespace of the value denoted by .x.y. The first . in a pathname is known as the root namespace for an evaluation context.

3.Rules

An Ivory system application is expected to be largely rule triggered. The reason for this is to de-couple state changes and dependent actions.

A simple ECA (Event Condition Action) approach is used where conditions and actions are combined in case expressions of functions with a common type signature known as rule methods.

(Rule -> (Exp *) -> ADS -> EventPhase -> Void)

There is a set of event types corresponding to external events and data store transactions. The most important of these are:

Events are always associated with a particular data store, and there is a simple built-in function to raise (or signal) them:
raiseEvent event::(Exp *) ads::ADS =
   let invokeMethod ref::Ref =
      case ref of {
         rule::Rule -> ((rule.method) rule event ads)::Void
      }
   in
      mapProcRefs invokeMethod (ads.ruleList)

This is for illustrative purposes only. In practice, there is an extra argument of type EventPhase that allows actions to be deferred until all rule methods have been evaluated.

Rules are associated with data stores by including them in the ruleList property of the host ADS object. Note that the same rule may apply to more than one data store.

mapProcRefs is a built-in function to apply a procedure successively to each element of a RefList (a vector of object references).

mapProcRefs proc !refs =
   let !doFrom !i proc !refs =
      if i < (length refs) then {
          proc (getAtRefList refs i);
          (doFrom (i + 1) proc refs)::Void
      }
   in
      doFrom 0 proc refs

This is usually implemented tail recursively as would be expected for a functional language. However, IvoryScript requires an explicit type constraint on tail recursive applications.

Pattern matching is used by the method property of a rule (typically a built-in function) to filter out particular events of interest. For the standard timer rule, this would involve invoking the appropriate operating system specific functions.

monitorTimer rule::Ref event ads::ADS =
   case event of {
      StartEvent _ -> ...
      -- Instantiate all timers in the store
      
      CreateObjectEvent timer::Timer -> ...
 	   -- O/S specific function to instantiate a timer

      UpdatePropertyEvent After timer::Timer #period
 	   -- O/S specific function to change the period of a timer
   }

4.Ivory Session

An Ivory session may be initiated in one of two ways:
  1. By submitting an initial script or byte code known as an order.
    This is a sequence of expressions, which are evaluated in order. The main purpose of which is to populate a store

  2. By attaching to a root persistent store.

In either case, if the root namespace contains an entry named start, it is evaluated.

5.Built-in Primitives

Built-in primitives are the means by which Ivory supports component-based application models. The principle is to identify and encapsulate common functions that can be used for as broad a range of applications as possible.


home

Last update: 11 October, 2005