PIXEL Programming Interaction with XML Environments and Languages

The aim of this research is to enable effective management of information by creation of a visual document, programming, and information management system and interface. The means for this is creation of a visual development environment for programming using XML based technologies. This will enable interactivity with and further development of this system by end-users such as administrative staff. An objective is to investigate use and improve visual environments and create proof of concept software for free distribution between academics, and amongst others, of this interactive environment. This will enable editing of the program structure using a visual representation of the software over the web that can be translated directly into code. Users will be able to change the program without ever needing to see the XML (eXtensible Markup Language) code. This will allow people to create interactive software that is well structured and searchable, and the best options for making this creation as straightforward as possible will evaluated. There are many techniques, tools and languages available for structuring and searching web pages, these will be applied to management of information.

Mayo and Steinberg [1] authored a cabinet office report for the UK Government recommending government partners with other providers to make use of User generated content and enable provision of information that helps achieve public policy objectives. A JISC (Joint Information Systems Committee) report explains the power of networks and how important it is for educational establishments to make proper use of them [2]. The flexibility of information representation languages could enable administrative users to maintain, adapt and extend a system themselves. This goes beyond open source development to enable development of communities of end-user programmers. Users can interact with XML via programs written in interpreted languages such as JavaScript. The user will not require any software except for a web browser as the development environment will be web-based.

XML is used for structuring information so different software systems can interpret it. This language can be used to structure web pages so that they can represent and link to programming objects. XML can be used as a way to manage workflow within and between organisations. XML provides the basis for creation of taxonomies and ontologies. Using Meta-tags defined with XML it is possible to create documents that define their own structure and to provide a consistent and understandable user interface. A further objective of this research is automating the conversion of information between formats, to assist organisations to manage their information.

Mechanisms for this Research
The intention is to prototype the creation an environment based on Semantic Web languages such as OWL (Web Ontology Language), RDF (Resource Description Framework) [3] and XML, not just for search but also combined into a comprehensive application that is usable for end-user programming of a large range of problems. To allow users to create Rich Internet Applications we intend to create a translator using XML or RDF/XML programming so the entire solution would be in XML based languages. This involves programming with Semantic Web languages rather than just using them for information representation. This will make the translation from visualisation to code easier and more reliable, and further improve the maintainability of the software created.

It is essential that new ways of enabling collaboration between all those involved in software creation and use are investigated. An important part of the research will be to investigate the use of Semantic Web languages for programming, rather than just for representation of information, this would expand on languages such as SPARQL (Simple Protocol and RDF Query Language) [3] and XQuery [4]. The use of Semantic Web languages for declarative programming can ease the translation between different representations of information, and ease interoperability between systems.

End-User XML Editing
An XML based drag and drop programming example has been created [5], this example was created with AJAX (Asynchronous JavaScript And XML). XML can be manipulated directly from the browser rather than requiring a specialist XML editor. This allows open standards platform independent end-user programming. The example is based on the Scand dhtmlxTree [6] and this makes it possible to enable many other programming actions such as adding and deleting nodes, and to create other editing controls. It will be possible to make use of XML based meta programming to allow end-user programming, Simkin [7] and Lemos [8] are examples of this approach. We intend to investigate and extend this approach to create meta language tools for programming, including for the display interface. Semantic languages provide a higher level declarative view of the problem to be modelled. We will develop declarative programming with XML based languages. Examples of research into XML as a programming language are AspectXML [9], and the Minimal Imperative Language XIM [10]. Interactive web programs can be created using scripting languages and XML combinations such as AJAX (Asynchronous JavaScript And XML) [11], which is a name for techniques to create highly interactive web pages. Web 2.0 technologies (often making use of Ajax) aim to reproduce on the web the functionality provided by office software and this can be extended to enable users to create their own programs using XML and a visual interface.

Even where documents are represented using XML or other structured languages, it is important to structure the contents and semantics using an ontology, Erdmann and Studer [12] experiment with this.

References
[1] Mayo E, Steinberg T, 2007, The Power of Information, Cabinet Office independent review - http://www.cabinetoffice.gov.uk/publications/reports/power_information/power_information.pdf?id=3965.
[2] JISC (Joint Information Systems Committee) Anderson P, 2007, Technology and Standards Watch What is Web 2.0? Ideas, technologies and implications for education - http://www.jisc.ac.uk/media/documents/techwatch/tsw0701b.pdf.
[3] World Wide Web Consortium (W3C), 2007, SPARQL Query Language for RDF, http://www.w3.org/TR/rdf-sparql-query/.
[4] McGovern J, Cagle K, Bothner P, Nagarajan V, Linn J, 2003, XQuery Kick Start.
[5] Drag and Drop, 2007, http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/samples/components.html.
[6] Scand dhtmlxTree, 2007, http://www.scbr.com/docs/products/dhtmlxTree/index.shtml.
[7] Whiteside S, 2007, Simkin the embeddable scripting language, http://www.simkin.co.uk/.
[8] Lemos M, 2007, MetaL: An XML based Meta-Programming language, http://www.meta-language.net.
[9] Peterson M D, 2005, O'Reilly XML.com - [Part 3] Assets, Atom Feeds, and AspectXML - The Triple Threat of Web Development? - http://www.oreillynet.com/xml/blog/2005/09/part_3_assets_atom_feeds_and_a.html.
[10] Bayram Z, Onder R, 2007, XSL Transformations A delivery medium for executable content over the Internet, http://www.ddj.com/web-development/198800555.
[11] Cagle K, 2006, AJAX on the Enterprise, AJAXWorld conference, October 4, 2006 - http://www.oreillynet.com/xml/blog/2006/10/ajax_on_the_enterprise.html.
[12] Erdmann M, Studer R, 1999, Ontologies as Conceptual Models for XML Documents, Proceedings of the 12th Workshop on Knowledge Acquisition, Modelling and Management.

More Information

Ajax/Web2.0 - http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/ajax.htm.

Drag and Drop Programming - http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/ajax.htmDragandDropProgramming.

RDF, OWL, Semantic Web - http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/RDF/RDF.htm.

XML - http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/XML/XML.htm.

XML Programming - http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/XML/XML.htmProgrammingwithXML.

User-Driven Model-Driven Programming

A model-driven approach is important for enabling of user-driven programming. User-Driven Model-Driven Programming is the way to make it possible for a much wider range of people to do their own programming. This combines the approaches of End-User Programming, the Semantic Web and Modelling. The mechanism for this is program transformation, translating from user view to model to code. The diagram below shows this approach.

User Driven Model Driven Programming

Ways my research is pursued in order to make User Driven Programming possible are :-

1 Semantic Web and Web 2.0 approach to enabling User Generated Content - http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/ajax.htm.
2 User Centric Extensions to UML (Unified Modelling Language) e.g. http://www.himalia.net/ this approach also ventures into 1 and 3).
3 Visual Programming Extensions to spreadsheet type formulae based modelling, an example is http://wiki.vanguardsw.com/ enabled using approach 1.

This research links with the approach of enabling User Generated Content and providing a Visual Programming System. While my thesis concentrates mainly on 1 and 3, 2 is just as important and is referenced.

So there is considerable overlap between these three approaches and they must be integrated within interdisciplinary research to enable user driven programming. One approach to this is a Semantic User Interface, this is explained here -http://www.himalia.net/docs_semanticUI.html. This can enable Drag and Drop programming http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/ajax.htm#DragandDropProgramming that combines the benefits of all three research approaches. The important factor is to connect the user interface with the underlying code, so the two share the same structure and users can properly see how their actions can change the underlying code.

The next step is to make possible collaborative user-driven programming by sharing the visualisation of models across computer networks and between collaborators. In order to make user driven modelling and programming possible, it is essential that a communication mechanism is established, which allows users to generate changes and receive changes generated by the modelling system.

My pages on subjects related to this are -

End-User Programming - http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm.

Language and Tool Mapping - http://www.cems.uwe.ac.uk/~phale/#LanguageToolMapping.

Model-Driven Programming -http://www.cems.uwe.ac.uk/amrc/seeds/softwareengineering.htmModelDrivenProgramming.

Modelling and Semantic Web -http://www.cems.uwe.ac.uk/amrc/seeds/ModellingSemanticWeb.htm.

Semantic Web - http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/RDF/RDF.htm.

Systems Enginering and Simulation

The intention of this project is to create a way for non programmers to create software in high level open standards based declarative languages built with Semantic Web technologies. The reasoning behind this is that if Semantic Web languages can represent data, they can also represent programs, as programs are just a specialised kind of data. The non programmer would program with this declarative language by means of a diagrammatic visualisation.

Many people would like to make greater use of computer technology but are hampered by the need to learn programming languages if they are to fully interact with software. Instead they are limited to the use of certain generic features that are provided for them. A further constraint is the cost of software, and for this project we will develop free software and encourage a community of end-user developers, and modellers.

We aim to provide Semantic Web based modelling and simulation tools that are usable and programmable by end-users. This is in order to ease the difficulties involved in translating requirements between software experts and domain experts. Our research will develop, extend, and combine existing research in provision of web page development tools for end-users, meta-programming, and web based modelling and simulation tools. The diagram below explains the research area we intend to explore.

This research will be part of co-ordinated efforts to enable end-user programming for knowledge management (including e-learning), modelling and decision support, and simulation. Therefore the research will concentrate mainly on translation for simulation and enabling non programmers to create web-based simulation systems. This is illustrated in the figure below.

The main research area will be the interface between Meta-Programming, Modelling and Simulation, and Semantic Web Model Creation, shaded in the figure below. This could allow end-users to develop their own Semantic Web based simulation and modelling tools using a graphical visual interface.

A simple illustration of the techniques that can be used to further this research area is a demonstrator we completed for meta-programming of XML (eXtensible Markup Language) based drag and drop trees. Python is used as a translator between the XML representation of the trees and interactive graphical representations of them. This allows open standards platform independent end-user programming. Such techniques could be used with other Semantic Web based information representations based on languages and structures such as XML, RDF (Resource Description Framework), and OWL (Web Ontology Language), and provision of other controls. These could then be used as graphical components of a simulation system made available over the web. This demonstrator furthered the research of Anderson and Krause [1]. Whiteside also used XML based meta programming to allow end user programming of games with Simkin [2]. Semantic languages provide a higher level declarative view of the problem to be modelled.

Standardisation in XML/RDF enables use of declarative rules for web services. Rules play an important role in artificial intelligence, knowledge-based systems, and for intelligent agents. To allow information sharing and reuse, interoperability, and collaboration an ontology centric approach can be used [3]. Ontologies are defined by Gruber in [4], and he also examines how equations and quantities can be represented in an ontology. Extending such work can enable functions and calculations to be represented in OWL and called by a Semantic Web based programming language.

References

[1] Sample code using drag & Drop with a tree - http://lists.wxwidgets.org/archive/wxPython-users/msg11332.html - Drag and Drop contributed by Sam Anderson, reposted by Dirk Krause.

[2] Simkin - http://www.simkin.co.uk/Links.shtml.

[3] COSMOA: An Ontology-Centric Multi-Agent System For Coordinating Medical Responses To Large-Scale Disasters. Bloodsworth, P., Greenwood, S., 2005. AI Communications Vol 18 (3) Agents Applied in Health Care pp 229-240.

[4] Toward Principles for the Design of Ontologies Used for Knowledge Sharing - Gruber T. R. 1993, http://www2.umassd.edu/SWAgents/agentdocs/stanford/onto-design.pdf - In Formal Ontology in Conceptual Analysis and Knowledge Representation, edited by Nicola Guarino and Roberto Poli, Kluwer Academic Publishers, in press. Substantial revision of paper presented at the International Workshop on Formal Ontology, March, 1993, Padova, Italy. Available as Technical Report KSL 93-04, Knowledge Systems Laboratory, Stanford University.

Other Pages on this subject

http://www.cems.uwe.ac.uk/amrc/seeds/ModellingSemanticWeb.htm.

http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/RDF/RDF.htm.

http://www.cems.uwe.ac.uk/amrc/seeds/softwareengineering.htm.

E-Learning Provision using Semantic Web Technologies

It would be useful to build a Semantic Web system for modelling and education mainly in the engineering and science fields. This would involve enabling a Semantic Web modelling infrastructure that could be the basis of future research in learning systems. In order to achieve these aims applications could be invesitgated and reviewed that assist in model building. The system created would also be used to promote engineering to a skeptical public who see the profession as poorly paid and dirty, (House of Lords Select Committee on Science and Technology report science and society), and remote from the public (Canavan et al).

E-Learning Research Area - The aim of this research is to try to bring together the areas of E-Learning, End-User Programming and the Semantic Web. So to achieve this, the area marked in yellow is examined.

With the development of technologies under the broad terms of Semantic Web and Web 2.0 there are opportunities to establish a highly interactive web based learning environment. The constructionist approach to understanding problems is to learn about them by modelling them. This approach can be used in combination with research into enabling end user programming techniques for modelling in order to provide an environment for non programmers to model their problems. Visualisation and interaction will provide rapid feedback that gives a powerful representation of the environment to be modelled.

Many people would like to make greater use of computer technology but are hampered by the need to learn programming languages if they are to fully interact with software. Instead they are limited to the use of certain features that are provided for them. A further constraint is the cost of software, and for this project we will develop free software and encourage a community of end-user developers, and modellers.

The aim is to create a software development environment that enables people to customise their own software solution. This is an alternative to provision of software as a finished article that can't be changed. It is intended that the learning software can be customised without requiring programming expertise.

This research will be part of our co-ordinated efforts to enable end-user programming for knowledge management (including e-learning), modelling and decision support, and simulation. Therefore the research proposed in this bid will concentrate mainly on translation for knowledge management and enabling academics and students to create e-learning systems. This is illustrated in the figure below.

The intention is to create an online community that can provide free software for modelling and education. The advantages of open source collaboration are that as well as allowing researchers to co-operate and work together where they share an interest; it will also allow the untapped potential to be developed of those who do not have an official research position. This includes students, people employed outside the academic environment, retired people and amateurs who have useful expertise. Astronomy, for example, has harnessed skills of this very diverse range of people to make new discoveries (Mutchler and Schenk). The expertise provided by anyone involved would be in providing feedback on usefulness, or ease of use of software, as well as actual involvement in software development. This means feedback would be sought from users of software even if they were not software experts. Meetings and forums could be arranged, both for web based and real world participation. Researchers would benefit the community by providing modelling and education tools online and for libraries. This is different from other open source communities in that the software could be easier to use and develop, and so involve those who have not previously been capable of participating in software development.

It is important to provide an alternative way of representing models, which does not require the user to write code. The tool created must make it easy to interact with and change the models, and easy to share information with others. This can also involve use of editing tools such as wikis (Protégé Community Wiki),(Vanguard Software Modelling Wiki), (Visual Knowledge), blogs, and semantic web editors (Amaya),(Quint, V., Vatton, 2004 and 2005) to allow discussion and explanation of the models.

There is an urgent need for Semantic Web tools to illustrate the benefits this technology can provide for education, this paper (EASE: The European Association Semantic Web Education) (Diederich et al) explains this need. There is also a need for education in the use of Semantic Web technologies for industry. Some Semantic Web tools are available, explained by EASE and in the (Jena User Conference) or being developed at present, but they are still difficult for people to make use of as they take a good deal of development expertise. REASE (the repository of EASE for learning units) provides a way to find and create learning materials for industrial applications of Semantic Web technologies.

References

Select Committee on Science and Technology Third Report Chapter 2: Public Attitudes and Values - Attitudes to engineering 2.39 - http://www.publications.parliament.uk/pa/ld199900/ldselect/ldsctech/38/3804.htm.

Canavan B, Magill J, Love D, A Study of the Factors Affecting Perception of Science, Engineering and Technology (SET) in Young People (2002), International Conference on Engineering Education, August 18-21, 2002, Manchester, U.K.

Amaya, 2007. Welcome to Amaya - W3C's Editor/Browser http://www.w3.org/Amaya/.

Diederich, J, Nejdl, W, Tolksdorf R, 2006, EASE: The European Association for SemanticWeb Education, SWET2006 Beijing, China.

Jena User Conference, 2006, Bristol, UK - http://blog.myspace.com/UK%20http://jena.hpl.hp.com/juc2006/proceedings.html - Proceedings (2006).

Mutchler M, Schenk H, Amateur Astronomers and the Hubble Space Telescope - http://www-int.stsci.edu/~mutchler/amateur.html.

Protégé Community Wiki - User Driven Progamming - http://protege.cim3.net/cgi-bin/wiki.pl?UserDrivenProgramming.

Quint, V., Vatton, I., 2004. Techniques for Authoring Complex XML Documents, In: DocEng 2004 - ACM Symposium on Document Engineering Milwaukee October 28-30 http://wam.inrialpes.fr/publications/2004/DocEng2004VQIV.html.

Quint, V., Vatton, I., 2005. Towards Active Web Clients, In: DocEng 2005 - ACM Symposium on Document Engineering Bristol United Kingdom 2-4 November 2005 - http://wam.inrialpes.fr/publications/2004/DocEng2004VQIV.html.

REASE the repository of EASE for learning units http://ubp.l3s.uni-hannover.de/ubp.

Vanguard Software Modelling Wiki - http://wiki.vanguardsw.com/bin/browse.dsb?dir/Engineering/Aerospace/ - Engineering - Aerospace

Visual Knowledge - http://www.visualknowledge.com/ - Semantic Wiki.

My pages on this subject

My Home Page is - http://www.cems.uwe.ac.uk/~phale/.

My E-Learning Page is - http://www.cems.uwe.ac.uk/~phale/ELearning.htm.

My Semantic Web Page is - http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/RDF/RDF.htm.

Drag and Drop Programming

This post examines how drag and drop programming and other forms of interactive software development can assist end-users to program via the web. AJAX (Asynchronous JavaScript And XML) software is used to provide a Web 2.0 style interactive interface. This can be linked to applications and to Semantic Web information.

Lieberman [1] blames end user programming difficulties on hard to understand programming languages and techniques and argues for visualisation and translation to code to enable end-user programming. To make end-user programming easier it is possible to develop high level visualised templates and translate these into program code. The Dagstuhl report [2] argues that existing programming languages are not sufficiently dependable for end-users to reliably use. De Souza [3] argues that the goal of human-computer interaction (HCI) will evolve from making systems easy to use to making systems that are easy to develop. A template system will assist in this. Ko [4] explains that end-user programmers must be allowed to focus on their goals, and an important part of the solution is to visualise the whole program execution not just the output, so it is necessary to show the user the whole program flow not just text based bug reports. A simple illustration of the techniques that can be used to further this research area is a demonstrator for meta-programming of XML (eXtensible Markup Language) based drag and drop trees [5], this example was created with AJAX (Asynchronous JavaScript And XML), a Java applet or Python were other options investigated. The code acts as a translator between the XML representation of the trees and interactive graphical representations. This allows open standards platform independent end-user programming. The example is based on the Scand dhtmlxTree [6] and this makes it possible to enable many other programming actions such as adding and deleting nodes, and to create other controls. Repenning [7] argues that visual programming languages using drag and drop mechanisms as a programming approach make it virtually impossible to create syntactic errors, allowing for concentration on the semantics", and Rosson [8] also advocates this technique. Such techniques could be used with other Semantic Web-based information representations implemented with languages and structures such as XML, RDF (Resource Description Framework), and OWL (Web Ontology Language), and provision of other controls. These controls could then be used as graphical components of a simulation system made available over the web. As well as being used for web-based visual programming an environment such as this could also be employed as an interface to PC based software, or as a translator between systems. Semantic languages provide a higher level declarative view of the problem to be modelled. Coutaz [9] explains that "An interactive system is a graph of models related by mappings and transformations." This would fit in well with the structure of RDF, which is also a graph structure.

It is important to investigate new ways of enabling collaboration between all those involved in software creation and use. The use of Semantic Web languages for declarative programming can ease the translation between different representations of information, and ease interoperability between systems. This translation or 'Program Transformation' allows for writing in one representation or language, and translating to another. This is particularly useful for language independent programming, or for high level and end-user translation to a language more easily interpreted by computer systems. The solution to many interoperability and software problems involves programming with Semantic Web languages rather than just using them for information representation. This will make translation for interoperability easier and more reliable, and further improve the maintainability of software systems. The research will involve using and building a visualised Semantic programming layer on languages such as AspectXML, XForms, SPARQL, and XQuery all explained in [10] and Meta languages [11][12] to create software and to build an environment for high level end-user programming. This programming environment can be used for creating programs and an environment for end-user programming. The environment can be computer language and system independent as one representation can be translated into many computer languages or Meta languages. Tools such as Amaya [13] can be used for creating and editing Semantic Web applications and documents. This research is a test case for an approach of collaborative end-user programming by domain experts. The end-user programmers can use a visual interface where the visualisation of the software exactly matches the structure of the software itself, making translation between user and computer, and vice versa, much more practical.

To enable generic end-user programming functionality it is important to develop ‘information representation languages’ based on Semantic Web declarative programming languages. Standardisation in XML/RDF enables use of declarative rules for web services. This environment also has to provide a visual development interface for end-users in a similar way to that of Unified Modeling Language (UML) for professional developers. Repenning [7] and Engels [14] argue this.

References

1] Lieberman, H., 2007. End-User Software Engineering Position Paper. In: End-User Software Engineering Dagstuhl Seminar - http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1092.
[2] Burnett M M, Engels G, Myers B A, Rothermel G, 2007, End-User Software Engineering Dagstuhl Seminar, http://eusesconsortium.org/docs/dagstuhl_2007.pdf.
[3] De Souza, C., 2007. Designers Need End-User Software Engineering. In: End-User Software Engineering Dagstuhl Seminar.[4] Ko, A. J., 2007. Barriers to Successful End-User Programming. In: End-User Software Engineering Dagstuhl Seminar - http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1083.
[5] Drag and Drop, 2007, http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/components.html.
[6] Scand dhtmlxTree, 2007, http://www.scbr.com/docs/products/dhtmlxTree/index.shtml.
[7] Repenning, A., 2007. End-User Design. In: End-User Software Engineering Dagstuhl Seminar - http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1099.
[8] Rosson, M. B., A., 2007. Position paper for EUSE 2007 at Dagstuhl. In: End-User Software Engineering Dagstuhl Seminar. - http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1094.
[9] Coutaz, J., 2007. Meta-User Interfaces for Ambient Spaces: Can Model-Driven-Engineering Help?. In: End-User Software Engineering Dagstuhl Seminar - http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1082.
[10] Programming with XML, 2007, http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/XML/XML.htm#ProgrammingwithXML.
[11] Dmitriev S, 2007, Language Oriented Programming: The Next Programming Paradigm - http://www.onboard.jetbrains.com/is1/articles/04/10/lop/.
[12] Mens K, Michiels I, Wuyts R, 2002, Supporting Software Development through Declaratively Codified Programming Patterns, Expert Systems with Applications Vol 23.
[13] Quint V, Vatton I, 2005, Towards Active Web Clients, DocEng Bristol United - http://wam.inrialpes.fr/publications/2005/DocEng05-Quint.html.
[14] Engels, G., 2007. Model-Driven Development for End-Users, too!? In: End-User Software Engineering Dagstuhl Seminar http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1085.

Other Information -
Drag and Drop Programming - http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/ajax.htm#DragandDropProgramming.
Example Interfaces - http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm#ExampleInterfaces.