Visualisation and Interaction for Modelling

Eng and Salustri (2006) outline the role of computers in aiding decision making, and explain that the human mind is the best tool for making decisions. They explain that visualisation systems must help people use the information access capabilities of computers. Eng and Salustri refer to a dimension from “tacit to articulatable” knowledge. So the research for this thesis aims to use the layered Semantic architecture described by Berners Lee et al. (2000) and discussed by McGuinness (2003) relating to this diagram :-

Layered Architecture, sourced from McGuinness (2003) and Berners-Lee (2000)

This improves translatation between the layers to enable human/computer translation. This approach is intended to improve interaction rather than enable computing decision making through artificial intelligence; the emphasis is on decision support for design and manufacture. The detail of this approach and the methodology for automating translation for users is explained in here - http://www.cems.uwe.ac.uk/~phale/#ResearchMethodology. Such techniques as genetic algorithms are outside the scope of this thesis. Instead the emphasis is on clear visualisation, interaction and translation.

This translation code reproduces a taxonomy/ontology and makes it available for modelling/programming systems. This taxonomy/ontology is a copied subset of the main ontology produced as an instance of the main ontology according to model builder choices and for the modelling/programming purposes of that model builder.



Recursive Translation - Automated Copying from ontology to modelling system

Also the translation can link different ontologies/taxonomies together when they are required in order to solve a problem. So the approach is to gather information from ontologies/taxonomies as required for solving a problem as specified by the model builder. This is tested and applied to engineering modelling. An open source approach can be combined with use of open standards ontologies as was advocated by Cheung (2005).

References

Berners-Lee, T., (2000) Semantic Web on XML – Slide 10 [online]. Available from: http://www.w3.org/2000/Talks/1206-xml2k-tbl/slide1-0.html [Accessed 26 June 2008].

Cheung, W. M., Maropoulos, P. G., Gao, J. X., Aziz, H., 2005. Ontological Approach for Organisational Knowledge Re-use in Product Developing Environments. In: 11th International Conference on Concurrent Enterprising - ICE 2005, University BW Munich, Germany.

Eng, N., Salustri, F. A., 2006. "Rugplot" Visualization for Preliminary Design. In: CDEN 2006 3rd CDEN/RCCI International Design Conference University of Toronto, Ontario, Canada.

McGuinness D. L., 2003. Ontologies Come of Age. In: Dieter Fensel, Jim Hendler, Henry Lieberman, and Wolfgang Wahlster, ed. Spinning the Semantic Web: Bringing the World Wide Web to Its Full Potential. MIT Press, 2003 [online]. Available from: http://www-ksl.stanford.edu/people/dlm/papers/ontologies-come-of-age-mit-press-(with-citation).htm [Accessed 26 June 2008].

Who will write tomorrow's code?

This article on the BBC Technology Website by Bill Thompson illustrates aspects of the history of programming and examines the availability of future programmers.

This is at - http://news.bbc.co.uk/1/hi/technology/7324556.stm.

"Bill Thompson describes the history of "personal computers like the IBM PC, home systems like the Spectrum and of course to the BBC Microcomputer," - "characterised by programmability, by the ability of users to write their own code".

"Just like EDSAC the BBC Micro was open to developers to work on, coming with the BASIC programming language and even a schematic showing the circuitry inside for those more interested in hardware than software."

"Along with the Spectrum and even the ZX81 it was a computer that encouraged those using it to learn to program for themselves."

"The BBC offered support with television programs, magazines with long BASIC programs to be typed in and improved upon were published, and there was a sense that understanding computers meant being able to do more than just run applications.

"Today, with over two billion PCs, three billion mobile phones and well over a billion internet users the ways in which information and communications technologies shape the modern world are obvious."

"The revolution succeeded, and we now live in the wired world of digital data, fast networks and computerised systems."

Bill Thompson argues "we need good programmers for the UK's software industry and at the moment we do not have enough of them."

Since the eighties and computers like the BBC Micro the emphasis on programming computers has reduced and use of applications is the main aim of most people. Although with Web 2.0 there is more User Generated Content created there are still limits to the customisation and development of applications by end-users. If the ease of use/programming of early BBC micros was brought to current applications and an easier transition from using to programming, this would be a powerful end-user programming environment. Then greater numbers of computer literate users would take on programming.

More information from my website on these subjects is available at - http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm and http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/EndUserHistory.htm.

History of End User Programming

1960s

In the 1960s Dartmouth BASIC programming language [7] was designed and implemented at Dartmouth College by John Kemeny and Thomas Kurtz. Over time BASIC became a popular language for home users, and business use, it introduced many people to programming as a hobby or career. Many of the modern concepts of computer graphics, dynamic objects and object oriented programming were prototyped by Ivan Sutherland in 1963 in Sketchpad [13][14]. In the mid 1960s Seymour Papert, a mathematician who had been working with Piaget in Geneva, came to the United States where he co-founded the MIT Artificial Intelligence Laboratory with Marvin Minsky. Papert worked with the team from Bolt, Beranek and Newman, led by Wallace Feurzeig that created the first version of Logo [25] in 1967. In the late sixties Alan Kay [2][3][17] used the term 'personal computer' and created a concept prototype, the FLEX Machine, he also envisaged a 'Dynabook' machine, the sketches for this look very similar to the laptop computers of recent years. The Simula [28] language was developed by Ole-Johan Dahl and Kristen Nygaard and this included Object-Oriented concepts. Douglas Engelbert's worked on a project to augment the human intellect, as part of the Augment [8] project he demonstrate Hypertext and video conferencing.

1970s

Alan Kay joined the Xerox Palo Alto Research Center (PARC) [17][19] California in 1971. Throughout the seventies the group at PARC led by Dr. Kay developed an integrated programming language and programming environment called Smalltalk [10]. In the early seventies the Alto personal computer was created at the PARC. The Alto eventually featured the world's first What-You-See-Is-What-You-Get (WYSIWYG) editor, a commercial mouse for input, a graphical user interface (GUI), and bit-mapped display, and offered menus and icons, and linked to a local area network. The Alto provided the foundation for Xerox's STAR 8010 Information System. There was still a need to find a common use for a personal computer that would increase the demand for it. In 1978, Harvard Business School student, Daniel Bricklin, came up with the idea for an interactive visible calculator. Bricklin and Bob Frankston then co-invented the software program VisiCalc [1]. VisiCalc was a spreadsheet, and the first 'killer' application for personal computers as this application provided a justification for using personal computers as a productive tool.

1980s

During the 1980s ownership of personal computers became increasingly popular and many home users programmed using BASIC. In the early eighties IBM developed the first personal computer built from off the shelf parts (called open architecture) [15]. This included a command line operating system written by Microsoft and the Microsoft BASIC programming language. Apple developed the GUI further for the Lisa [5] that later became the Macintosh (Mac). The IBM style PC became most popular for business applications, while the Apple Mac was often used for Desktop publishing.

1990s

End User Programming research has continued to the present day. Research has continued in techniques of Visual Programming [9] e.g. Alice [4], Programming by Example [2][21], programming with automated assistance [20], and Natural Language Programming [27]. Squeak and Croquet[6] have developed from the early work in Smalltalk.

Tim Berners-Lee [23] developed HyperText Markup Language (HTML), and has been involved with the World Wide Web Consortium (W3C) [29] in developing standards base languages for the Web. This has encouraged the growth of the 'Semantic Web' [11] which allows both humans and computers to search and interact with pages more and so encouraged the development of interactive web pages and communities.

2000s

Recent, present and future research can enable the use of semantic web technologies, (developed from HTML by Tim Berners-Lee [23] and others), to enable End User Programming. This fusion of research and technologies is illustrated by Henry Lieberman's home page [12] which has explanations of both areas of research. Examples of this fusion include Protégé [22], Jena [16], TopBraid Composer [24], and OpenCyc [18]. Information about these technologies is available in my semantic web page - http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/RDF/RDF.htm. A related development is that of web 2.0. Visual development environments based on AJAX (Asynchronous JavaScript And XML) [26] aim to reproduce on the web, the functionality provided by office tools such as Excel (which is often used as an End User Programming Environment). Information about Ajax and Web 2.0 is available in my Ajax/web2.0 page - http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/ajax.htm.

References

1. A Brief History of Spreadsheets - http://dssresources.com/history/sshistory.html - Decision Support System Resources - by D. J. Power, Editor, DSSResources.COM.

2. Alan Kay - http://www.acypher.com/wwid/FrontMatter/index.html - Watch What I Do - Programming by Example.

3. Alan Kay ETech 2003 presentation - http://www.lisarein.com/alankay/tour.html - Lisa Rein's Tour Of Alan Kay's Etech 2003 Presentation.

4. Alice v2.0 - http://www.alice.org/ - Learn to Program Interactive 3D Graphics.

5. Apple Lisa - http://fp3.antelecom.net/gcifu/applemuseum/lisa2.html - The First Affordable GUI - Lisa 1 Jan-83 Jan-84, Lisa 2 Jan-84 Apr-85.

6. Croquet - http://www.opencroquet.org/ - a new open source software platform for creating deeply collaborative multi-user online applications.

7. Dartmouth BASIC - http://en.wikipedia.org/wiki/Dartmouth_BASIC - Wikipedia.

8. The Demo - http://sloan.stanford.edu/mousesite/1968Demo.html - Stanford University.

9. Dmoz Open Directory Project - http://dmoz.org/Computers/Programming/Languages/Visual/ - Visual Languages - Programming Languages Reference - Visual Languages.

10. The Early History Of Smalltalk by Alan Kay - http://www.smalltalk.org/smalltalk/TheEarlyHistoryOfSmalltalk_II.html - 1967-69--The FLEX Machine, a first attempt at an OOP-based personal computer - Alan Kay - Smalltalk.org.

11. Fifteen Years of the Web - http://news.bbc.co.uk/1/hi/technology/5243862.stm - Internet Timeline - BBC Technology.

12. Henry Lieberman - http://web.media.mit.edu/~lieber/ - Research Scientist - MIT Media Laboratory.

13. History of HCI - http://www.idemployee.id.tue.nl/g.w.m.rauterberg/presentations/HCI-history - Key systems, people and ideas - Presentation by Matthias Rauterberg.

14. History of HCI - Sketchpad (1963) - http://www.idemployee.id.tue.nl/g.w.m.rauterberg/presentations/HCI-history/sld020.htm - Ivan Sutherland - MIT Lab - Presentation by Matthias Rauterberg.

15. Inventors of the Modern Computer - http://inventors.about.com/library/weekly/aa031599.htm -The History of the IBM PC - International Business Machines.

16. Jena - http://jena.hpl.hp.com/juc2006/proceedings.html - First Jena User Conference - Proceedings.

17. Kyoto Prize Laureates 2004 - http://www.kyotoprize.org/commentary_kay.htm - 2004 Kyoto Prize Laureates - Dr. Alan Curtis Kay (U.S.A., b. 1940) - Computer Scientist, President, Viewpoints Research Institute.

18. OpenCyc - http://www.opencyc.org/ - OpenCyc.org - General knowledge base and commonsense reasoning engine.

19. Palo Alto Research Center (PARC) - History - http://www.parc.xerox.com/about/history/default.html - PARC History.

20. The Programmer's Apprentice - http://portal.acm.org/citation.cfm?id=87912&dl=ACM&coll=GUIDE - The ACM Digital Library.

21. Programming by Example - http://web.media.mit.edu/~lieber/PBE/index.html.

22. Protege - http://protege.stanford.edu/ - Protégé Home - Ontology Development Environment.

23. Tim Berners - http://www.w3.org/People/Berners-Lee/Lee - Tim Berners-Lee.

24. TopBraid - http://www.topbraidcomposer.com/ - Semantic Modeling Toolset - Visual modeling environment.

25. What is Logo? - http://el.media.mit.edu/Logo-foundation/logo/index.html - MIT Logo Foundation, What is Logo.

26. Wikipedia - http://en.wikipedia.org/wiki/Ajax_(programming) - Ajax (programming).

27. Wikipedia - http://en.wikipedia.org/wiki/Natural_language_and_computation - Natural language processing.

28. Simula - http://en.wikipedia.org/wiki/Simula - Simula.

29. World Wide Web Consortium (W3C) - http://www.w3.org/ - Leading the Web to Its Full Potential....

Useful End User Programming Links

"A History of Haskell: being lazy with class", Paul Hudak (Yale University), John Hughes (Chalmers University), Simon Peyton Jones (Microsoft Research), Philip Wadler (Edinburgh University), http://research.microsoft.com/~simonpj/papers/history-of-haskell/index.htm - The Third ACM SIGPLAN History of Programming Languages Conference (HOPL-III) San Diego, California, June 9-10, 2007.

Alan Blackwell - University of Cambridge - Human Computer Interaction - End User Programming.

BBC Technology news - Free tool offers 'easy' coding - http://news.bbc.co.uk/1/hi/technology/6647011.stm - A free programming tool that allows anyone to create their own animated stories, video games and interactive artworks has been developed - Jonathan Fildes - 14 May 2007.

Celebrating the creator of Cobol - http://news.bbc.co.uk/1/hi/technology/6168489.stm - BBC News - Mark Ward December 11th 2006.

The Centre for Advanced Learning Technologies - The Centre for Advanced Learning Technologies - studying the impact of new media and technologies on the business environment.

Computer History Museum - Exhibits - Timeline.

Computer Languages History - http://www.levenez.com/lang/ - Computer Languages Timeline - Éric Lévénez. - O'Reilly Poster based on Éric Lévénez diagram.

Constructivist Computer Assisted Learning: Theory and Techniques - http://www.ascilite.org.au/conferences/adelaide96/papers/21.html - Barney Dalgarno - Information Services Division - University of Canberra - The changes that have occurred in accepted approaches to teaching and learning in recent years have been underpinned by shifts in psychological and pedagogical theory, culminating in moves towards a constructivist view of learning.

Dmoz Open Directory Project - Programming Languages - Programming Languages Reference - Alphabetic List of Programming Languages - Definitions and Links.

DSpace - http://www.dspace.org/ - The DSpace digital repository system captures, stores, indexes, preserves, and distributes digital research material.

Euses End Users Shaping Effective Software research collaboration - Welcome to EUSES - Research Collaboration.

Euses Presentation - End User Programming - Invited Research Overview - Brad Myers, Andrew Co, Margaret Burnett - Carnegie Mellon, Oregon State Universities.

Generative Programming - Generative Programming - Methods, Tools, and Applications - Krzysztof Czarnecki and Ulrich W. Eisenecker - Addison-Wesley, June 2000.

Hackety Hack - http://hacketyhack.net/ - In this century, you may have dozens of programming languages lurking on your machine. But how to use them?? A fundamental secret! Well, no more. We cannot stand for that. Hackety Hack will not stand to have you in the dark!!

History of Computing - http://www.ieuc.org/end-user-computing/references/notes/HistoryofComputing.html - One of the best works in this regard can be found in the volumes devoted to The History of Programming Languages. - The Insititute for End User Computing.

History of End User Programming - Article - Peter Hale.

History of Haskell - http://haskell.org/haskellwiki/History_of_Haskell.

How the internet transformed business - BBC Business - By Steve Schifferes Business editor, BBC News website.

How the Spectrum began a revolution - http://news.bbc.co.uk/1/hi/technology/6572711.stm - In April 1982 a small British company, lead by Sir Clive Sinclair, launched the ZX Spectrum computer and sparked a revolution. - 23 April 2007 - BBC News Technology.

How the web went world wide - BBC Technology - Mark Ward Technology Correspondent, BBC News website.

IBM developerWorks Interviews: Rod Smith - http://www-128.ibm.com/developerworks/podcast/dwi/cm-int062806.html - IBM vice president of Emerging Internet Technologies on the business of watching, encouraging, and leveraging new technologies.

IBM QED Wiki - IBM eyes programming for the masses - By Martin LaMonica - CNET News.com.

I think, therefore I Woz - http://news.bbc.co.uk/1/hi/technology/6077374.stm - BBC News - Technology - Steve Wozniak,Apple - 25th October 2006.

In pictures: Commodore computers - http://news.bbc.co.uk/1/hi/in_pictures/6454113.stm - The rise, fall and rise again of gaming icon Commodore - 15 March 2007.

ISTI-CNR, Pisa, Italy - Model-based Tools for Pervasive Usability - Fabio Paternò.

John Backus (1924-2007) - John W. Backus, who built and led the IBM team that created Fortran, the first widely used programming language, which helped to open the door to modern computing, died on 17 March at his home in Ashland, Oregon, USA. He was 82.

Journal of Visual Languages and Computing - Journal Home Page - Elsevier.

Network of Excellence on End User Development - Network of Excellence on End User Development - EUD-Net

Oregan State and Houston University - Automatic Generation and Maintenance of Correct Spreadsheets - Martin Erwig, Robin Abraham, Irene Cooperstein, Steve Kollmansberger.

Palo Alto Research Center (PARC) - http://www.parc.com/ - Palo Alto.

Raskin Center - http://rchi.raskincenter.org/index.php?title=Home - Exploting New Interface Directions.

Science Museum - http://www.sciencemuseum.org.uk/collections/subject_themes/computing.asp - Computing and Information Technology.

Semantic Information Processing - Semantic Information ProcessingMarvin L. Minsky - The MIT Press.

Software Abstractions - Resources and Additional Materials - Book with sample chapters online - Daniel Jackson.

Socratic Arts - http://www.socraticarts.com/ - Online learning services.

Software componentry - http://en.wikipedia.org/wiki/Software_componentry - Wikipedia - Software componentry.

The beauty of software - British Computer Society Turing Lecture March 2007 Grady Booch - Full write up - http://www.bcs.org/server.php?show=ConWebDoc.10367 - This year's Turing Lecture was given by Grady Booch under the title 'The promise, the limits, the beauty of software.' - 13 March 2007

The Dangers of End-User Programming - Portland State University - Warren Harrison.

The Geometer's Sketchpad:Programming by Geometry - http://www.acypher.com/wwid/Chapters/13Sketchpad.html - R. Nicholas Jackiw and William F. Finzer - from Watch What I Do: Programming by Demonstration - edited by Allen Cypher co-edited by Daniel C. Halbert, David Kurlander, Henry Lieberman, David Maulsby, Brad A. Myers, and Alan Turransky.

The History of Computer Programming Languages - http://www.princeton.edu/~ferguson/adw/programming_languages.shtml - Stephen Ferguson - Princeton University Library.

The History of Computing Project - http://www.thocp.net/.

The Institute for End User Computing, Inc. The Chronicles of End User Computing... http://www.ieuc.org/home/chronicles.html as edited on Saturday, January 22, 2005.

The Institute for End User Computing, Inc. The IEUC Homepage - http://www.ieuc.org/home.html - as edited on Wednesday, May 17, 2006.

The Institute for End User Computing - The Market's Failure to Meet End User Needs - http://www.ieuc.org/home/market-failure.html.

The Magic of the 80's - http://www.ieuc.org/end-user-computing/references/notes/themagicofthe80s.html - For a wonderful cultural history of the early days of the PC revolution, see S. Levy, Hackers : heroes of the computer revolution - The Insititute for End User Computing.

Twenty five years of the IBM PC - BBC News - Technology.

UK home computer pioneer honoured - http://news.bbc.co.uk/1/hi/technology/6217447.stm - BBC News - British technology pioneer Andrew Hopper becomes a CBE in the New Year Honours list.

Universidad Autónoma de Madrid - Dr José A. Macías - publications - Research - End User Development (EUD).

University of the West of England - UWE Student Project - http://www.cems.uwe.ac.uk/amrc/seeds/Web%20Semantic/Index.html - Investigating and implement the idea of 'ModConsWest' (Modelling and Constructionism with Web based E-Learning Semantic Tools)" - Lee Ediagbonya and Awaab Eltahir.

Where does the web go from here? - BBC Technology - Bill Thompson.

Useful Publications

A Computer Program to Model and Stimulate Creative Thought, Smith, D. C. (1977), Basel: Birkhauser. 187p.

A History of Haskell: being lazy with class, Paul Hudak (Yale University), John Hughes (Chalmers University), Simon Peyton Jones (Microsoft Research), Philip Wadler (Edinburgh University), http://research.microsoft.com/~simonpj/papers/history-of-haskell/index.htm - The Third ACM SIGPLAN History of Programming Languages Conference (HOPL-III) San Diego, California, June 9-10, 2007.

Estimating the Numbers of End Users and End User Programmers, Scaffidi, C., Shaw, M., Myers, B. (2005). IEEE Symposium on Visual Languages and Human-Centric Computing, (VL/HCC'05): 207-214 Dallas, Texas.

Example-based Programming: a pertinent visual approach for learning to program (2004) - University of Poitiers - Nicolas Guibert - Patrick Girard - Laurent Guittet - Proceedings of the working conference on Advanced visual interfaces - Pages: 358 - 361 - ISBN:1-58113-867-9.

History of Programming Languages, Bergin T J, Gibson R G, 1996, Volume 2, ISBN-10: 0-201-89502-1; ISBN-13: 978-0-201-89502-5.

Interaction-Oriented Software Development (2001) Huhns M N, International Journal of Software Engineering and Knowledge Engineering 11 3 259-277.

Model-based tools for pervasive usability, 2005, Paterno Fabio, Interacting with Computers 17, 291-315.

The Programmer's Apprentice, 1990, Rich C, Waters R C, The ACM Digital Library - ISBN:0-201-52425-2.

Watch What I Do: Programming by Demonstration - Cypher, A, 1993, MIT Press, ISBN:0262032139.

http://www.acypher.com/wwid/ - Watch What I Do: Programming by Demonstration - The entire text of this book is included on this web site. Access it through the Table of Contents.

http://www.acypher.com/wwid/FrontMatter/index.html.

Your Wish is My Command: Giving Users the Power to Instruct their Software - http://web.media.mit.edu/~lieber/Your-Wish/ - Henry Lieberman, editor.

I am a Researcher in the final year of my PhD. I specialise in applying Semantic Web techniques. My current research is on a technique of 'User Driven Modelling/Programming'. My intention is to enable non-programmers to create software from a user interface that allows them to model a particular problem or scenario. This involves a user entering information visually in the form of a tree diagram. I am attempting to develop ways of automatically translating this information into program code in a variety of computer languages. This is very important and useful for many employees that have insufficient time to learn programming languages. I am looking to research visualisation, and visualisation techniques to create a human computer interface that allows non experts to create software.

I am a member of the Institute for End User Computing - http://www.ieuc.org/home.html.

My Home Page is http://www.cems.uwe.ac.uk/~phale/.

A web page for this article is at http://www.cems.uwe.ac.uk/amrc/seeds/PeterHale/EndUserHistory.htm.