Open Standard Layered Architecture for computer to human translation

This diagram shows a central infrastructure of an open standards layered architecture. This enables interoperability at the Computer to computer layer. This gives advantages to developers for maintenance and re-use. This infrastructure aids translation from computer and developers upwards, to end users. Visualising the model/program structure translated upward from code to a navigable interactive visualisation enables accessibility, thus assisting with modelling and end user programming. This infrastructure that aids computer to computer interoperability thus also aids human to human collaboration. This all aids ease of use and re-use of models/programs also.

So far this translation has been enabled upwards from computer to human. Future research could involve translation from human to computer and interaction to make this an iterative, interactive life-cycle process.

Translation between models/programs

As well as user to computer translation, an important area of research is similar semi-automated translation between computer programs and models. Much of the structure of programs/models is the same regardless of what particular programming language is used. Making translations of a program/model available on multiple systems by translation from one to the other eases maintenance and re-use and widens availability. Also involving users in such a process would enable sharing and understanding of models/programs, so aiding Decision Support.

Collaboration and Process Modelling in Engineering and Business - Conclusions

This post concludes the series that explains my research in this subject, and points towards how the research can be used in engineering and business, and how it can be furthered by others. This explains what was achieved in provision of modeling and decision support systems that could automate user-computer translation, and interact with users such as engineers to help them model problems and make appropriate decisions.

To make this research into modeling for engineering practical for a wide range of computer literate non-programmers, sustained further research is needed in the areas of visualisation, interaction, modeling, end user programming, and transformation as well as the links between these areas.

The need is for a methodology for creation of systems to enable more collaborative approaches to modeling by domain expert end users. This combined with visualisation would allow engineers to model problems accurately. This can be made possible by provision of systems that may not fulfil all of the requirements of the domain experts because this would be difficult or impossible, but instead allows the domain experts, e.g. engineers to customise this themselves and help build the system they need and that could be changed as required. Alternatives to the current approach to software development are required. Modeling languages can be used as an interface to an end user programming environment. Transformation from a model building environment to program code was investigated.

Even if programming is made easier, only a proportion of people would actually be interested or capable of doing this. But there is still an advantage to colleagues such as people in the same team or department as an end user programmer, even if they are not undertaking programming themselves. Then all in the team have much closer access to someone, the end user programmer, who understands their, and the team's tasks, requirements, and projects. This closes the gap between those producing software systems, and those who require the software. This also makes it easier to iterate through solutions and solve problems more quickly and collaboratively.

Experienced programmers can build a modeling environment that can then be used by non programmers to create models or solve other software problems. This was achieved for the DATUM (Design Analysis Tool for Unit-cost Modelling) project with Rolls-Royce, and the modelling environment created was used by their engineers. Collaboration, simulation and modeling have been investigated to determine the requirements for future research in modeling of problems.

This should allow translation from a model based representation of software to the actual software. This can involve semi automatically producing software for a Semantic website from visual representations of the problem. The core of this modeling infrastructure is automated generation of models created with World Wide Web Consortium (W3C) standards based languages, and the visualization of information represented in such W3C standard ways.

This research investigated alternative approaches to software development, which give users greater involvement, and can actually be used in combination. This partially automates the process of software creation via a collaborative process and equation tree that maps the problem structure, and user interface creation by providing a means to manage a diagrammatic and/or tree based representation.

User Driven Modelling Explanation - Wing Spar

Figure 1 illustrates the implementation of the translation stages. Step 1 is creation of the ontology, which is then translated to the decision support and modelling tool (Vanguard System) for Step 2. Step 2 is illustrated to the right, and this shows colour coding of the taxonomies (sub ontologies) that make up the ontology e.g. parts, processes, and materials. Step 3 involves translations to visualisations for the web (using Semantic Web formats) and alternative representations. Step 3 can also produce program and/or meta-program code.





Figure 1. Stepped Translation and Visualisation

Figure 2 demonstrates the ontology translated via Step 2 into XML for Step 3 visualisation in Flash (see reference end of article). This creates a tree with a three dimensional look, colour and shading, and interactive repositioning of nodes to make it intuitive and assist in navigation. When a node is chosen, this is moved to the centre of the display and all the other nodes are moved or rotated to position themselves in relation to it.

Figure 2. Flash interface for navigating exported XML tree

Figure 3 shows the view resulting from choosing the 'SparPart Definition'. This shows the parents, children, siblings, and contents of that node. It also allows navigation to any of the related nodes.

Figure 3. Flash viewing of Spar Part Definition node

Figure 4 is produced via an automated conversion from a tree representation of the spar component. The interface demonstrates modelling of information within a browser; ‘Periphery’, ‘Area’, ‘Raw Volume’, ‘Finished Volume’, ‘Part Width’ and ‘Part Height’ are all calculated dynamically. This calculation is in response to changes the user makes to the attributes on the left; as these changes are made the diagram changes in response.


Figure 4. Interactive Spar Diagram (SVG)

Reference

Rhodes, G., Macdonald, J., Jokol, K., Prudence, P., Aylward, P., Shepherd, R., Yard, T., 2002. A Flash Family Tree, In: Flash MX Application and Interface Design Flash MX Application and Interface Design. ISBN:1590591585. [online]. Available from: http://www.friendsofed.com/book.html?isbn=1590591585.

User Driven Modelling Explanation - Rectangle

This simple model explains all the implementation of translation steps.

Step 1

A rectangle is defined with attributes for length and width, shown in the screenshot below :-

Figure 1 - Rectangle Definition in Ontology

Width is then defined as 2m.

In the next figure another class is created for calculation of area.

Figure 2 - Rectangle Area Attribute

In figure 1 'Area' was assigned a value of 'Length' * 'Width'. This is a simple equation that will be used to calculate the result. This illustrates how modelling calculations are performed. They are all defined by equations that relate attributes of the taxonomy/sub ontology. The taxonomy can be read by the decision support system in Step 2.

Step 2

For Step 2 the decision support/modelling system reads the ontology, performs the calculation and deals with units giving the result for area as 8 metres squared. This is shown in Figure 3 below :-

Figure 3 - Step 2 - Translation and Calculation

The modelling system can then output the results to web-based visualisations, or to program/meta-program code.

Step 3

As well as showing the model itself on the web it is possible to translate the model results into other representations and visualisations, so making it as widely accessible as practical. Figure 4 shows the result model translated into XML and visualised as a tree view on the web. The automated menu provides for some of McGuinness (2003) 7 uses of simple ontologies :-
2. site organization and navigation support
3. expectation setting
5. browsing support
The use and visualisation of Semantic Web languages at all stages in the translation process facilitates 6. ‘Search support’, and 7. ‘sense disambiguation support’; the structure of the ontology is visualised in order to enable users to determine the context of terms.

Figure 4 Results Output as Tree (XML based)

Figure 5 shows an output SVG rectangle diagram that includes interactivity; this has been translated from the tree/graph-based representation. The input values used for the calculation and the diagram itself can be changed via an automatically produced user interface that is related to the taxonomy structure. These changes cause the shape representation and the area to be recalculated.

Figure 5 - CAD type interface and User Interaction and Calculation

A demonstration and related pages can be accessed below -

http://www.cems.uwe.ac.uk/~phale/RectangleDemo/RectangleDemo.viewlet/RectangleDemo_viewlet_swf.html

Background information to justify why this approach should be researched is here - User Driven Modelling Background.
A Flash movie that presents this concept using a simple example - User Driven Modelling Demonstration.
Translation demonstration (A Flash Player is required to view this) - RectangleDemo

- Explanation - http://www.cems.uwe.ac.uk/~phale/RectangleDemo/RectangleDemo.viewlet/RectangleDemo_launcher.html
Demonstraion of User-Driven Modelling/Programming process for translating from taxonomy based to diagram based rectangle representation

Simple SVG Example of Rectangle as demonstrated in the Flash Movie - Internet Explorer Version - Requires SVG player - SVG Rectangle Internet Explorer.
Simple SVG Example of Rectangle as demonstrated in the Flash Movie - Mozilla Firefox - SVG Rectangle Mozilla Firefox.

Reference

McGuinness, D. L., 2003. Ontologies Come of Age. In: Dieter Fensel, Jim Hendler, Henry Lieberman, and Wolfgang Wahlster, ed. Spinning the Semantic Web: Bringing the World Wide Web to Its Full Potential. MIT Press, 2003.

Automated Production of Engineering Models

This example demonstrates the research I have undertaken into automating the model creation process for engineering process models.

This example is of manufacturing a cube, where the cost of manufacture depends mainly on the size of the cube, its wall thickness, what material is used, and what process is used. This is a simple example, a real world example involving hand layup of a spar (wing component) proved to be too complicated for ease of demonstration explanation, especially over the web.

An ontology (held in Protege http://protege.stanford.edu/) is used to represent the common information needed for all cube models, (and other models) produced. The ontology is broken down into sub ontologies for parts, materials, processes, consumables, resources, and tooling. These are colour coded in the visualisation/interactive model to ease understanding.

Code written for this research is used to recursively read the Protege tree (via nested SQL calls), and reproduce the tree in the modelling system (Vanguard System http://www.vanguardsw.com/products/vanguard-system/). The modelling system enables calculations, and extra code written for this research also allows choices to be made by the user/modeller.

This simple example enables the user/modeller to make choices of the material, process, consumables, resources, and tooling to be used, for the manufacture of this cube. In this example choices were made for material - Aluminium, and process - Rivetting.

Cube Model

This model is then output to an online representation (this works in Internet Explorer only, a version for Mozilla Firefox is also in progress). The IE version requires the Adobe SVG player, currently downloadable, and free).

The online SVG (Scalable Vector Graphics) representation provides an interactive CAD type representation of the component, that can be manipulated, to change size and wall thickness. The wall thickness is indicated by the inner dotted cube. As the size or wall thickness of the cube is altered, immediate feedback is provided. Alterations can be made with the up/down buttons. Also if these values, or the Aluminium cost per metre cubed are changed, the calculated parameter and cost values change automatically, in response.

The SVG representation shown below can be found at http://www.cems.uwe.ac.uk/~phale/SVGCubeExample/CubePartDefinitionwithCosts.htm.

Cube Model - Translated to SVG

http://www.cems.uwe.ac.uk/~phale/SVGCubeExample/CubePartDefinitionwithCosts.htm.

It is also possible to zoom in and out of the diagram, and move it around the screen.

As well as engineering models, I have experimented with economics models, and with translating either type of model to Java, and Java applets. Below is a screenshot of a simple economic model translated and visualised as an interactive Java applet.

Consumption Function translation from modelling system into Java

The models created as part of this research are available at - http://www.cems.uwe.ac.uk/~phale/EconomicModels/ModelsVisualised.htm.

Automation of knowledge representation and modelling

Knowledge management is a combination of the domains information is applied to and the technology used to represent this knowledge. The main domains I'm involved for representing this knowledge are engineering (process and design), and economics. The Semantic Web is key to this, but the Semantic web also involves many back end computing technologies such as databases/ontologies, and software tools/translation to represent this semantics/knowledge. To test this approach an application is necessary, to show benefits of the Semantic Web/knowledge management, and a way of testing it on the above domain. For my work, this problem used for testing is modelling, but in order to enable modelling using computing technologies, it is essential to make the modelling tools available to as great a range as possible of end-users. So this brings in the need for end-user programming, as to create models, it is necessary to program.

So my research area is as in the shaded ares of the diagram below -

Knowledge management and modelling are time consuming and difficult, this is only partly because of the technologies used, but my research focusses mainly on the technologies. When there are problems in keeping up with the needs of people, such as productivity problems in manufacturing, one of the solutions has been to mechanise this. So the mechanism I'm working on applies this idea to software. I'm working on enabling modelling, and programming for modelling, and what is necessary to ease this problem is provision of a machine. This machine undertakes translation and automated production of models from information. This is similar to the ideas behind virtual machines, interpreters, compilers, and UML tools, but there is more interaction with the users in the tools created for this research. So this enables customisation, at the same time as automation.

The models created are at - http://www.cems.uwe.ac.uk/~phale/EconomicModels/ModelsVisualised.htm.

More explanation of this research is at - http://sites.google.com/site/userdrivenmodellingprogramming/.

User Driven Modelling - Applications

This visual modelling is similar to collaborative spreadsheet modelling, but for these research models the visualisation and the code structure are exactly the same. This means users effectively can see an audit trail at all times in the modelling, and that the gap between those that program/model, and users is reduced. This makes both computing automated translation, and human collaboration/domain translation easier. This research is mainly aimed at users whose models/programs are too complex for spreadsheets. This means that themselves and other modellers/users lose track of information. These modellers/users often do not have the time and/or facilities for the programming tasks they need.

Automated translation and visualisation of models is essential as if they are manually copied by humans, there are almost certain to be errors in the copying of any large model to a different system. For such models an error anywhere in the model makes the result wrong. Visualising over collaborative networks, and of the whole structure also allows those responsible for checking and maintenance of each individual part of it to spot any mistakes, so this again acts as an effective audit trail.

Demonstration interactive models are shown at - http://www.cems.uwe.ac.uk/~phale/EconomicModels/ModelsVisualised.htm.

Research Methodology - Automated Model Production

Models are constructed and translated in a different way from object oriented programming. There is no clear distinction between classes and objects, as a class only becomes an instance gradually as translations are made from step 1 to 3, and as the models are visualised, interacted with, and represented to users. In the early research an ontology was translated and created from a C++ program, and also from a database. Now, models are created in the Vanguard System (http://www.vanguardsw.com/products/vanguard-system/) modelling tool, with information imported to this from a Protégé ontology, via the nested SQL queries. Vanguard System performs the calculations necessary for the economic model. After inferencing/calculation a result ontology is created for step 3. The step 3 result ontology/taxonomy is created by manually coding, then a system is created to reproduce and output this code automatically. This translation is then tested on many different models to ensure it is generic.

The diagram below examines many of the ways translation could be used for User Driven Modelling/Programming. An example of the repeatable process of this design is that the ‘System Translator Program’ created in Step 1 produces a new ‘System/Translator Program’ in Step 2 which creates a Visualisation. This technique is used to translate an ontology to a CAD style diagram using a second stage of translation. The second ‘System Translator Program’ could also have created a ‘Model/Program’, ‘Meta Program’ or translate to an ‘External Application’. So, this is not an exhaustive diagram, as many types of translation not shown on this diagram would be possible. Another option is that Step 1 could be repeated to translate between ontologies.

Translation Process for User Driven Modelling/Programming (UDM/P)

The ontology represents the relationships between nodes in a machine independent way, so this makes it possible to translate the information into Meta languages via recursive querying. For Step 2 translation, SQL (Structured Query Language) is then used to access the underlying database representation of the ontology. These SQL calls cycle recursively through all the relationships and write out result code for each node, and each relationship automatically. The translation code reads node names and node types (e.g. class, attribute) so it can make an appropriate SQL call for each node, and make a copy in another language or system, then look for children or attributes. This allows any tree to be represented in computer languages. Then recursive routines write the programming language to be output.

More information is available at http://sites.google.com/site/userdrivenmodellingprogramming/index.

Models produced so far are available at - http://www.cems.uwe.ac.uk/~phale/EconomicModels/ModelsVisualised.htm.

Economic Models 2

These economic models are constructed and translated in a different way from object oriented programming. There is no clear distinction between classes and objects, as a class only becomes an instance gradually as translations are made, and the models are visualised and represented to users.

At present the main focus is on extending the translation to interactive representations in Java and JavaScript. These will allow users to amend the parameter values of models and to see the result recalculated.

Once all or most of the translations are fully working, I'll concentrate on adding multiple models. Each model could then be translated automatically to many different visualisations/representations. I'll concentrate mainly on economic models, but might also eventually include other kinds of equation based models.

Economic Models - previous post - http://userdrivenmodelling.blogspot.com/2008/11/economic-models.html.

Economic Models Example -
http://www.cems.uwe.ac.uk/~phale/EconomicModels/ModelsVisualised.htm.

Java Applet Example - http://www.cems.uwe.ac.uk/~phale/EconomicModels/Bized/ConsumptionFunctionVanguardOutput/ConsumptionFunctionVanguardOutputInteractive/build/ConsumptionFunctionModelJavaAppletVanguardInteractive.html.

Economic Models

In order to prove the concept that User Driven Modelling is applicable to domains outside engineering, I'm developing economic models using the same kind of translation technique as I used for engineering models. Models are created in the Vanguard System (http://www.vanguardsw.com/) modelling tool, and can be imported to this from Protégé http://protege.stanford.edu/, via nested SQL queries. Vanguard System performs the calculations necessary for the economic model.

The next step is to visualise the Models in the web browser, and in various languages, to show the concept of multiple language implementations being created from one model. These multiple language implementations all share the same semantics and provide a tree based representation of this semantics.

These are demonstrated at http://www.cems.uwe.ac.uk/~phale/EconomicModels/ModelsVisualised.htm, different representations are provides, so that hopefully at least one representation is accessible to the various web browsers. So far there is an XML, HTML, and Java Applet representation. My intention is to extend both the representation of models and the number and type of models represented, until there is a large grid of models, and representations. Options for extending the representation are to JavaScript, SVG, RSS, RDF, and meta programs, and to increase, the interactivity/editability of the models.

This technique should allow automated creation of many models and language representations of them automatically, using one set of model code outputters/translators.

I'm getting the economic models from Biz/ed - http://www.bized.co.uk/educators/he/spreadsheet/section_1.htm. Eventually I might extend this to other types of models as well as engineering and economic models.

The implementation of the models is at http://www.cems.uwe.ac.uk/~phale/EconomicModels/ModelsVisualised.htm.

User Driven Programming/Modelling, and wider participation

A user-driven approach to visualisation, modelling, and programming can improve software and ease it's creation and maintenance. Translation then becomes posible from the user's representation to a software representation, and from design to implementation.

For User Driven Programming, 'user' can mean individual and/or organisation. The technique can be applied soonest to modelling/programming problems that require calculation. E.g. scientific, engineering, and business problems. The basis of the technique is enabling the information to be represented in a structured format e.g. XML (eXtensible Markup Language), RDF (Resourece Description Framework), RSS (this can stand for 3 different things - http://en.wikipedia.org/wiki/RSS_(file_format), and OWL (Web Ontology Language). Visualisation and interaction with the information can be enabled in a standardised way because the information is structured. This enables diagrammatic editing for modelling/programming. Careful structuring of the information and models/programs simplifies the modelling/programming process. A translation process from abstract/domain models to code can then be provided using recursive techniques.

As the structure and design and editing process for these models/programs can be standardised this enables a common meta-programming based methodology. Visualisation of this methodology to allow User Driven participation could enable wider participation than is currently possible for open source development, which is limited to more technical users. Visualisation at different levels of abstraction from domain to software could enable end-users, designers, and software developers to communicate in a semi-structured way in order to establish procedures for ensuring software is produced that end-users want to use.

Open Standard Representation of Rules and Equations

SWRL

Miller and Baramidze (2005) explain and SWRL (Semantic Web Rule Language). Zhao and Liu (2008) examine the need for sharing product information between partners as a product model, and how agreement through ontologies, Semantic Web, and standards can assist this. Zhao and Liu examine mapping of STEP representations to ontology languages OWL - http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/RDF/RDF.htm#OWL and SWRL and how this benefits interoperability. Zhao and Liu are encoding STEP rules and executable statements into OWL and SWRL. Zhao and Liu also show a diagram (similar to this one of Berners-Lee) of the position of OWL and SWRL in a stack of standards from XML in the Syntax layer up to OWL/SWRL in the Logic/Rule layer of 'Semantics'.

SWRL (Semantic Web Rule Language) combining OWL and RuleML and its use in modelling will also be investigated. This could be used for formally specifying the construction of equations and rules in a model and the relationships and constraints between items represented in an equation. Miller and Baramidze (2005) explain the SWRL language. An editing facility to model these equations and constraints, so that errors could be prevented, would improve the usability of future visual modelling systems created. Support for SWRL in Protégé and other ontology based systems will assist with the construction of a modelling system with sophisticated editing of rules (Miller and Baramidze, 2005). Miller and Baramidze (2005) examine efforts to develop mathematical semantic representations above the syntactical representations of MathML. SWRL also has standardised arithmetic and comparison operators (Zhao and Liu, 2008). These languages should enable standardisation of the representation of mathematical expressions that relate nodes, and their values and expressions; this would seem to be a difficult problem as it needs a user interface that enables complex mathematical structures to be conveyed by language and/or diagrammatic visualisation.

References

Miller, J A., Baramidze, G., Simulation and the Semantic Web - 2005. - Proceedings of the 2005 Winter Simulation Conference.

Zhao, W. and Liu, J.K. 2008. OWL/SWRL representation methodology for EXPRESS-driven product information model Part I. Implementation methodology, Computers in Industry - Article in Press, Corrected Proof - Abstract - This paper presents an ontology-based approach to enable semantic interoperability and reasoning over the product information model. The web ontology language (OWL) and the semantic web rule language (SWRL) in the Semantic Web are employed to construct the product information model. The traditional modeling language called EXPRESS is discussed. The representation methodology for EXPRESS-driven product information model is then proposed. The key of the representation methodology is mapping from EXPRESS to OWL/SWRL. Some illustrated examples are presented. - Keywords - Product information model; OWL; SWRL; EXPRESS; Ontology representation. Zhao and Liu (2008) are encoding STEP rules and executable statements into OWL and SWRL.

RuleML

The research of the Rule Markup Initiative is explained here. Rules for the Web have become a mainstream topic since inference rules became important in E-Commerce and the Semantic Web, and since transformation rules were put into practice for document generation from a central XML repository. Rules have continued to play an important role in artificial intelligence, knowledge-based systems, and for intelligent agents. This is now combined with standardisation in XML/RDF enabling use of declarative rules for web services. The Rule Markup Initiative has taken steps towards defining a shared Rule Markup Language (RuleML), enabling both forward (bottom-up) and backward (top-down) rules.

Reference

RuleML - The Rule Markup Initiative http://www.ruleml.org/.

Representation of Equations

Gruber (1993) examines how equations and quantities can be represented in an ontology. Kamareddine et al. (2004) examine how mathematics can be represented in order to make it easier for people to enter well-formed mathematical expressions.

Shim et al. (2002) examine translation from a users' model to equations and explain "converting a decision-makers' specification of a decision problem into an algebraic form and then into a form understandable by an algorithm is a key step in the use of a model". For a practical example the figure below explains the concept for a simple example of the representation of the equation E=MC². This relationship can be defined by the user. Here this is achieved using an ontology tool (Protégé), and this definition can be read directly by Decision support software (Vanguard Studio) that can visualise the information and colour code it. For a more complex example a higher level user interface would be required to enable a user to define the problem, and a translation step to the computer readable model. Units have been left out as the type of equation used and values in it are not important to the concept. The software can translate the source model into a program and calculate results. The result program is then translated again into a result model defined using open standard languages such as XML, and Java for human friendly visualisations viewable as web pages/diagrams.

Translation Process - Equations

References

Gruber T. R. 1993, Toward Principles for the Design of Ontologies Used for Knowledge Sharing - http://www2.umassd.edu/SWAgents/agentdocs/stanford/onto-design.pdf - In Formal Ontology in Conceptual Analysis and Knowledge Representation, edited by Nicola Guarino and Roberto Poli, Kluwer Academic Publishers, in press. Substantial revision of paper presented at the International Workshop on Formal Ontology, March, 1993, Padova, Italy. Available as Technical Report KSL 93-04, Knowledge Systems Laboratory, Stanford University.

Kamareddine, F., Maarek, M., Wells, J. B, 2005, Toward an Object-Oriented Structure for Mathematical Text - http://www.macs.hw.ac.uk/~jbw/papers/Kamareddine+Maarek+Wells:Toward-an-Object-Oriented-Structure-for-Mathematical-Text:MKM-2005.pdf - Mathematical Knowledge Management, 4th Int'l Conf., Proceedings, LNCS. Springer-Verlag.

Quint, V., Vatton, I., 2004. Techniques for Authoring Complex XML Documents - http://wam.inrialpes.fr/publications/2004/DocEng2004VQIV.html - DocEng 2004 - ACM Symposium on Document Engineering Milwaukee October 28-30 - This paper reviews the main innovations of XML and considers their impact on the editing techniques for structured documents.

Quint, V., Vatton, I., 2005. Towards Active Web Clients - http://wam.inrialpes.fr/publications/2005/DocEng05-Quint.html - DocEng 2005 - ACM Symposium on Document Engineering - 2-4 November 2005 - Bristol, United Kingdom. - Recent developments of document technologies have strongly impacted the evolution of Web clients over the last fifteen years, but all Web clients have not taken the same advantage of this advance. In particular, mainstream tools have put the emphasis on accessing existing documents to the detriment of a more cooperative usage of the Web. However, in the early days, Web users were able to go beyond browsing and to get more actively involved.

Shim, J.P., Warkentin, M., Courtney, J. F., Power, D J., 2002, Past, present, and future of decision support technology - Decision Support Systems - Volume 33, Issue 2 , June 2002, Pages 111-126.

MathML

MathML is a specification for describing mathematics as a basis for machine to machine communication. It provides a way of including mathematical expressions in Web pages. This is explained at World Wide Web Consortium Math Home (2008). There is support for creation and editing of MathML documents in Amaya (Amaya, 2008), (Quint and Vatton, 2004 and 2005).

MathML and semantics built on this could assist in the process of translating equations/formulae into code by providing an open representation of functions as XML (eXtensible Markup Language). Functions entered by the model developer can then be translated to this open representation and translated to programming languages and/or read by programming languages.

Miller and Baramidze (2005) examine efforts to develop mathematical semantic representations above the syntactical representations of MathML.

References

Amaya - http://www.w3.org/Amaya/ - Welcome to Amaya - W3C's Editor/Browser - Amaya is a Web editor, i.e. a tool used to create and update documents directly on the Web. Browsing features are seamlessly integrated with the editing and remote access features in a uniform environment. This follows the original vision of the Web as a space for collaboration and not just a one-way publishing medium.

Miller, J A., Baramidze, G., Simulation and the Semantic Web - 2005. - Proceedings of the 2005 Winter Simulation Conference.

World Wide Web Consortium (W3C) Math Home, 2008. What is MathML? http://www.w3.org/Math/Overview.html.

Links

Coq proof assistant - http://coq.inria.fr/ - Coq is a formal proof management system: a proof done with Coq is mechanically checked by the machine.

End User Programming for Scientists: Modeling Complex Systems - http://drops.dagstuhl.de/opus/volltexte/2007/1077/ - Andrew Begel - Microsoft Research - In: End-User Software Engineering - Dagstuhl Seminar - Summary - http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=2007081 - Margaret M. Burnett, Gregor Engels, Brad A. Myers and Gregg Rothermel - From 18.01.07 to 23.02.07, the Dagstuhl Seminar 07081 End-User Software Engineering was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl.

EquMath.Net - http://equmath.net/ - EquMath is resource for math lessons from Algebra to Differential Equations!.

Equplus - http://equplus.net/ - Science and Math Equations.

Mathematica - http://www.wolfram.com/ - Wolfram Research.

Mathematical Functions - Interactive Graph - http://www.richtann.com/2dExamples/graph.html - Richard Tann UWE - An Example using Actionscript to model functions and dynamically redraw according to user interaction.

MathLang: experience-driven development of a new mathematical language - http://citeseer.ist.psu.edu/kamareddine04mathlang.html - F. Kamareddine, et al., 2004 - Abstract: In this paper we report on the design of a new mathematical language and our method of designing it, driven by the encoding of mathematical texts. MathLang is intended to provide support for checking basic well-formedness of mathematical text without requiring the heavy and dicult-to-use machinery of full type theory or other forms of full formalization. At the same time, it is intended to allow the addition of fuller formalization to a document as time and e ort permits.

MathWeb - http://www.mathweb.org/ - MathWeb.org supplies an infrastructure for web-supported mathematics.

Mizar Project - http://mizar.org/project/ - The Mizar project started around 1973 as an attempt to reconstruct mathematical vernacular in a computer-oriented environment.

OMDoc.org - http://www.omdoc.org/ - OMDoc.org: The OMDoc Portal - OMDoc is a markup format and data model for Open Mathematical Documents. It serves as semantics-oriented representation format and ontology language for mathematical knowledge.

OpenMath - http://www.openmath.org/ - OpenMath is a new, extensible standard for representing the semantics of mathematical objects.

Richard Tann UWE - http://www.richtann.com/2dExamples/graph.html - Mathematical Functions - An Example using Actionscript to model functions and dynamically redraw according to user interaction.

Toward an Object-Oriented Structure for Mathematical Text - http://www.macs.hw.ac.uk/~jbw/papers/Kamareddine+Maarek+Wells:Toward-an-Object-Oriented-Structure-for-Mathematical-Text:MKM-2005.pdf - Kamareddine, F., Maarek, M., Wells, J. B, 2005, Mathematical Knowledge Management, 4th Int'l Conf., Proceedings, LNCS. Springer-Verlag.

SRML

SRML - Simulation Reference Markup Language - http://www.w3.org/TR/SRML/ - W3C Note 18 December 2002.

SRML case study: simple self-describing process modeling and simulation - http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1371486 - Reichenthal, S.W. Boeing, Anaheim, CA, USA; 2004, Simulation Conference, 2004. Proceedings of the 2004 Winter - Volume: 2, pp 1461- 1466 - ISBN: 0-7803-8786-4.

Visual Diagrammatic Programming

With visual diagrammatic modelling it was possible to include one model within another as a software component, and demarcate responsibility for building, maintenance, and updating of each model. This was difficult using spreadsheets, and possible with non-visual programming though the link between individual responsibilities and code produce was not as clearly identified, because non-programmers cannot participate in code production. As an example, for cost modelling of an aircraft wing, different experts might build software models for wing spars, wing skins etc, and another expert might be responsible for the overall wing cost model. The wing spar and skins model would then be inserted into the overall wing cost model.

The techniques demonstrated in this thesis can aid progress towards accessing of data held using Semantic Web standards, and also other information that might be locked into particular systems such as databases, spreadsheets and enterprise systems. The translation and de-abstraction approach assists with enabling high level diagrammatic visualisations to be used and translated to computer queries. Programming using Semantic Web technologies can :-

* Assist with translating non-Semantic Web information into Semantic Web information.
* Assist in production of Semantic Web information by end-users by.
* Assist end-users to query non-Semantic Web information.

Further Visualisation Information is at - http://www.cems.uwe.ac.uk/amrc/seeds/Visualisation.htm

End User Programming Information is at - http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm

Visualising Reports

Many organisations produce text based reports from their IT systems. But text based reports don't always show information well enough for good decision making. Automated conversion of these reports into Semantic Web languages could assist greatly with this. So a translation process is required and can be implemented as part of an overall User Driven Modelling/Programming Approach. Once reports are converted to a standardised representation, hierarchical information can be represented as clickable trees and numerical representation as charts.

This makes it possible to customise outputs from existing IT systems and so allows an improvement in readability of information without major changes to the way it's produced. This could provide a large gain at little cost.

Examples of Visualisations are here - http://www.cems.uwe.ac.uk/amrc/seeds/Visualisation.htm#VisualisationExamples.

Chart/Graph type visualisations are here - http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/Charts/charts.htm.

User Driven Modelling Justification

There are many computer literate people who are experts in a particular domain. Given the opportunity they will develop their own software (Scaffidi, 2005). There is a need in many domains for people to create software such as engineering models, scientific taxonomies, business systems, and in the arts. Examples of software systems that enable creation of models without necessitating code writing are spreadsheets, visual web development systems, and database management systems. Crapo et al (2002) explain the need for a methodology for creation of systems to enable more collaborative approaches to modelling by domain expert end-users that make use of structured and visualised information. Peirce (1904) explained that systems can be represented logically by humans, and this applies to software development and information representation on computers. Thus it could be made possible to represent systems with natural and visual languages. Though Nurminen et al (2003) argue that people prefer usability over automation, it is possible to achieve both by providing a stepped translation capability that ensures users are constantly kept informed of the steps taken. This approach is advocated by (Johnson, 2004). Automated translation from a people-centred view of information to software for computers can aid in software development by domain expert end-users and help ensure they can communicate models directly to computers. Software developers can then concentrate mainly on creating systems that enable domain experts to develop software, rather than developing individual software models on behalf of domain experts, which is difficult and prone to communication problems.

References

Crapo, A. W., Waisel, L. B., Wallace, W. A., Willemain, T. R., 2002. Visualization and Modelling for Intelligent Systems. In: C. T. Leondes, ed. Intelligent Systems: Technology and Applications, Volume I Implementation Techniques, 2002 pp 53-85.

Johnson, P., 2004. Interactions, Collaborations and breakdowns. In: ACM International Conference Proceeding Series; Proceedings of the 3rd annual conference on Task models and diagrams Vol 86 Prague, Czech Republic. - http://portal.acm.org/citation.cfm?id=1045448&coll=GUIDE&dl=GUIDE&CFID=17319495&CFTOKEN=29205382&ret=1Fulltext [Accessed 17^th February 2008].

Nurminen, J. K., Karaonen, O., Hatonen, K., 2003. What makes expert systems survive over 10 years-empirical evaluation of several engineering applications. Expert Systems with Applications 24(2) pp 199-211. - http://citeseer.ist.psu.edu/630566.html [Accessed 17^th February 2008].

Peirce, C.S., 1906. Prolegomena to an Apology for Pragmaticism [online]. Available from: http://www.existentialgraphs.com/peirceoneg/prolegomena.htm [Accessed 17^th February 2008].

Scaffidi, C., Shaw, M., Myers, B., 2005. Estimating the Numbers of End-users and End-user Programmers. In: IEEE Symposium on Visual Languages and Human-Centric Computing, (VL/HCC'05): 207-214 Dallas, Texas. - http://www.cs.cmu.edu/~cscaffid/papers/eu_20050923_vlhcc.pdf [Accessed 17^th February 2008].

PhD Summary - User Driven Programming

The research question to be answered in this thesis is - To what extent is it possible to improve collaborative software development through the use of diagrams without requiring people to learn computer languages?

My research involves enabling computer literate people who aren't programmers to create software. The means for this is construction of visual diagrams that represent the task, in a similar way to how family trees and taxonomies can be represented as diagrams. The visual interface is available from a website and the research involves translating this to code. The aim is to enable programming without requiring people to learn computer languages. I am particularly interested in making it possible for a wider range of people to collaborate on software development.

The methodology for this is automation by translation from a human level of abstraction to software.

The theory behind this thesis is that it is possible to create a systematic diagrammatic representation of logical thought to enable representation of user's wishes to a computer.

C.S. Peirce (1906) stated in 'Prolegomena to an Apology for Pragmaticism' "Come on, my Reader, and let us construct a diagram to illustrate the general course of thought; I mean a system of diagrammatization by means of which any course of thought can be represented with exactitude". That is the purpose of this research, but to limit the scope and so make application of this theory testable the research is restricted mainly to engineers (because they often think in terms of diagrams) and to the domain of modelling (which often requires diagrams). So the aim is to apply the research first where it can have the most use and encourage others to expand it for other domains and other users. This research is intended to simplify computing for computer literate non-programmers, this includes many engineers. The main research area is enabling users such as engineers to model the problems they encounter in manufacturing and design. However, the wider aim is to prototype research for enabling a larger range of software users to model their problems. The intention is to create collaborative tools that allow users to develop software in a way they will be familiar with from their use of spreadsheets.

The hypothesis is that it is possible to create an end-user programming environment, usable by non-programmers, which can have a wide variety of uses. The aim of this research is to create a modelling system that can be edited by computer literate non-programmers, and so demonstrate an application of end-user programming that could be used in a more generic way. The possibilities for a generic user-driven programming environment will be explained. It is possible to create an end-user visual programming environment using Semantic Web technologies, especially for modelling of information, where this approach is well suited. All that is necessary is to link the information visually via equations, and store these results for reuse and collaboration. This can make translation from humans to computers easier and more reliable than current software systems and languages. The use of Semantic Web languages as programming languages would assist greatly with interoperability as these languages are standardised for use in a wide range of computer systems.

The problem examined in this thesis is that of allowing domain experts to create decision support software. This is a way of testing the assumption that these domain expert's thoughts can be represented in a systematic diagrammatic way, and then translated to computer code. The main emphasis is on engineers, who experience problems in creating and sharing their software. The alternatives they have for creation of software are spreadsheets, which do not have collaboration and information modelling abilities sufficiently built in, or complex software that needs considerable expertise to use, and often still has insufficient collaboration or information modelling capabilities.

Peirce, C.S. - 1906. Prolegomena to an Apology for Pragmaticism http://www.existentialgraphs.com/peirceoneg/prolegomena.htm

SWIG-UK Powerpoint Presentation - at Hewlett-Packard

I gave a SWIG-UK Powerpoint Presentation - at Hewlett-Packard Bristol UK - semantic web interest group - http://www.cems.uwe.ac.uk/~phale/InteractiveModellingandVisualisationofInformation.ppt or http://cems.uwe.ac.uk/~phale/InteractiveModellingandVisualisationofInformation.ppt - November 23rd 2007.

This was on Interactive Modelling of Information, Visualisation and Translation Between different software and ontology representations.

There were many interesting presentations and links to all the talks are at - http://swig.networkedplanet.com/special.html.

More information about my research can be found at http://www.cems.uwe.ac.uk/~phale/ or http://cems.uwe.ac.uk/~phale/.

I had a comment saying Links don't work unless I remove www from the url. I'm not sure why the link seems OK as it is from my computer but not others so have put both options.

User Driven Modelling

In order to make user driven modelling and programming possible, it is essential that a communication mechanism is established, which allows users to generate changes and receive changes generated by the modelling system.

Types of Change

There are two types of change that can be applied to the model driven programming system, User Generated, and Model Generated.

User Generated

Figure 1 shows a user initiating a change, which is to delete a node from the bottom left and attach a new node to a branch in the top tight. The tree is translated to structured text, and this is further translated to Code.

Figure 1 - User Generated Change

For the second user generated change shown in figure 2 an object represented by a tree is visualised as a diagram. The user can amend either the diagram or the tree, in either case the change is filtered to the alternative representation and translated to the structured text and code.

Figure 2 - User Generated Change, Alternative Interfaces

Model Generated

A model generated change is initiated by the model itself, which changes the code and the structured text in response to a calculation (that may have been requested by the user). The model passes a translated result tree to the user interface to let the user know that the recalculations have been finished, and give the user the results using a suitable visualisation. This is shown in figure 3.

Figure 3 - Model Generated Change

More Information on this research is available on my Website at http://www.cems.uwe.ac.uk/~phale/.

and on my End user programming page at http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm.

Translation for Visual End User Programming

Research Theory influencing this Translation Mechanism

The use of the Semantic Web in my thesis is to be a means for open standard representation of information (built on XML), transformation into different representations as required, and for provision of a high level interface as a tool for model creation, and translation to program code. An 'elaborator', is used, this is a translator that converts the diagrammatic representation of the problem into software code. Translations can be performed into any programming or meta-programming language or open standard information representation language, the visualisation of the model created can be displayed on the web. This translation builds on research in program and model transformation. The translation software performs transformations as required between different programming languages and visual model views. This has been prototyped, but it is important to further this research in order to establish a user base, and make the translation generic. Figure 1 shows the process.

Figure 1 - Translation Process

Implementation

Translation Process

This research involves finding alternative ways of representing models, which do not require the user to write code. The intention is to make it easier to interact with and change the models, and to share information with colleagues. The information used in the models resides in an ontology, and from this ontology models can be automatically produced via a recursive translation tool that has been prototyped.

The research for my thesis uses a technique of interpreting information in order to create decision support programs automatically in response to user choices. This technique is then extended for use in the automatic creation of programs in other computer languages and systems. This can be achieved by automated translation of the Vanguard Studio information into other languages. The basis of this is that elaborators are nodes in the tree, which are automatically created and dynamically write objects. This allows the wing box definition to be translated to the decision support system for costing and then to other software such as web pages for further processing or visualization. An open standard semantic editor Protégé created by Stanford University (2007) was used to structure this information into related taxonomies. This ontology holds the definitions of nodes representing information, and calculations to be performed. Taxonomies are created in Protégé for 'Parts', 'Materials', 'Consumables', 'Processes', 'Rates', and 'Tooling' for a prototype costing system. 'Parts' is the core taxonomy. New categories can be produced as required. Domain experts would edit the taxonomies; these experts can specify the relationships of classes and the equations to be used via a visual user interface in Protégé. These relationships are evaluated and translated to produce computer code. Figure 2 illustrates how code is produced from the semantic relationships.

Figure 2 - Translation Process Implementation

This model can be used as it is, or be a template for the generation of a further model(s). An example interface, a section from a model produced automatically, is shown in figure 3. This information is saved using a generic structure based on keys that define all relationships, into a relational database. This enables storage of hierarchical data in a relational database and also allows for separation of information into tables according to category, and the use of SQL (Structured Query Language) to automatically query and structure the information as required. Vanguards' tree based decision support tool Vanguard Studio (2007) reads this information and represents it as colour-coded nodes. The code written for this thesis automatically queries the taxonomies that make up the ontology and links the information as required for the model. The code builds in all the links required for the equations and thus links up information from different taxonomies, the information is colour coded according to which taxonomy it is from. This same code can be reused for any modelling problem, it builds the equations and follows the links to build each equation tree, and attach this to the rest of the tree. The decision support tool can perform calculations and so output results. Figure 3 shows how the decision support tool can automatically construct and represent a branch in the tree, visualize an equation and calculate a result. Red nodes represent processes, green nodes represent the part definition and magenta nodes represent resources. This illustrates how 3 taxonomies have been automatically linked because they are needed in this calculation. In this prototype hundreds of calculations have been related to each other, this example illustrates that 'Area' was also calculated, and that this forms part of the tree for the 'Hand Layup Tool Cleaning Cost', which in turn is passed into other calculations. Hundreds of calculations using information from all the taxonomies are linked as required in this costing example. The time taken to perform the translation from the ontology and to perform all the calculations is a less than a second.

Figure 3 Ontology to Model Conversion

References

Stanford University, 2007. Welcome to protégé - http://protege.stanford.edu/.

Vanguard Studio, 2007. Global Knowledge Portal http://wiki.vanguardsw.com/.

My Research - http://www.cems.uwe.ac.uk/~phale/.

Modelling - http://www.cems.uwe.ac.uk/amrc/seeds/Modelling.htm

Semantic Web Modelling - http://www.cems.uwe.ac.uk/amrc/seeds/ModellingSemanticWeb.htm

Visualization - http://www.cems.uwe.ac.uk/amrc/seeds/Visualisation.htm