Drag and Drop Visual Programming

2 main conclusions resulted from examining the history of end-user computing :-

• Research that created prototype systems for specialist users, school children, and other researchers and programmers, but failed to take make headway in the mass market can be reused with more up to date technology to assist development.

• More pragmatic research that involved creation of tools for the mass market, but which avoided more long term research issues can now be extended.

One mechanism for applying new technologies for use in end-user programming is drag and drop programming.

Repenning (2007) explains the need for enhancements to UML to aid end-user programming. Repenning also argues that "Visual programming languages using drag and drop mechanisms as the programming approach makes it virtually impossible to create syntactic errors." So "With the syntactic challenge being - more or less - out of the way we can focus on the semantic level of End-User Programming." This can make a high level model driven approach to production of better models possible. Rosson (2007) also explains about creation of a web-based drag and drop interface.

Ways that research is pursued in this thesis in order to make User Driven Programming possible are:-

1. Semantic Web and Web 2.0 approach to enabling User Generated Content.

2. User Centric Extensions to UML (Unified Modelling Language) e.g. (Vernazza, 2007) this approach also ventures into 1 and 3).

3. Visual Programming Extensions to spreadsheet type formulae based modelling enabled using approach 1.

Approach 1 is illustrated by Yahoo Pipes (2008) which provides for drag and drop editing of visual components that can combine, sort, and filter RSS data sources in order to automate web application development. This enables modelling of information, and using such diagrammatic programming combined with enabling of calculations would combine these three ways of producing applications. Yahoo Pipes enables creation of modelling applications with Semantic Web information, so this could assist in providing more incentive for provision of Semantic Web information and applications to use it.

So there is considerable overlap between these three approaches and they must be integrated within interdisciplinary research to enable user driven programming. One approach to this is a Semantic User Interface; this is explained in Vernazza (2007). This can enable Drag and Drop programming that combines the benefits of all three research approaches. The important factor is to connect the user interface with the underlying code, so the two share the same structure and users can properly see how their actions can change the actual code. This enables collaborative User-Driven programming.

References

Pipes and Filters for the Internet - http://radar.oreilly.com/archives/2007/02/pipes-and-filters-for-the-inte.html - O'Reilly - Feb 7 2007 - Yahoo!'s new Pipes service is a milestone in the history of the internet. It's a service that generalizes the idea of the mashup, providing a drag and drop editor that allows you to connect internet data sources, process them, and redirect the output.

Repenning, A., 2007. End-User Design. In: End-User Software Engineering Dagstuhl Seminar.

Rosson, M. B., 2007. Position paper for EUSE 2007 at Dagstuhl. In: End-User Software Engineering Dagstuhl Seminar.

Vernazza, L., 2007. Himalia: Model-Driven User Interfaces Using Hypermedia, Controls and Patterns In: IFAC/IFIP/IFORS IEA Symposium - Analysis, Design, and Evaluation of Human-Machine Systems Seoul, Korea - September 4-6th 2007 - International Federation of Automatic Control.

Yahoo Pipes [online]. Available from: http://pipes.yahoo.com/pipes/ [Accessed May 22nd 2008].

This is a Yahoo Pipes example created for the subject of Web 2.0/AJAX -

Yahoo Pipes RSS Feed for Web 2.0 - http://pipes.yahoo.com/pipes/pipe.info?_id=3e30625ebd44f24ff969ae5eef724425

An explanation of how to access and use the pipe created from a web page is at - Code Explanataion/Tutorial Freshblog - Blogger Hacks, Categories, Tips & Tricks - http://blogfresh.blogspot.com/2007/03/pipes-json-and-code-for-your-website.html.

Another article I wrote on Drag and Drop programming is at http://ezinearticles.com/?Drag-and-Drop-Programming&id=671578 and information and examples at http://www.cems.uwe.ac.uk/amrc/seeds/Ajax/ajax.htm#DragandDropProgramming

Dagstuhl Seminar End-User Software Engineering - Part 4 - Final Part

End-User Software Engineering - Dagstuhl Seminar - Summary - http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=2007081 - PDF Abstracts and links to papers - http://eusesconsortium.org/docs/dagstuhl_2007.pdf - Margaret M. Burnett, Gregor Engels, Brad A. Myers and Gregg Rothermel - From 18.01.07 to 23.02.07

In 'Meta-User Interfaces for Ambient Spaces: Can Model-Driven-Engineering Help?' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1082 Joëlle Coutaz explains how Model Driven Engineering and Service Oriented Architecture can be combined -

"Our approach to UI plasticity brings together MDE (Model Driven Engineering) and SOA (Service Oriented Architecture) within a unified framework that covers both the development stage and the runtime phase of interactive systems."

Coutaz also explains that -

"An interactive system is a graph of models related by mappings and transformations."

This would fit in well with the structure of RDF (Resource Description Framework) a Semantic Web standard which is also a graph structure.

In 'Model-Driven Development for End-Users, too!?' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1085 Gregor Engels of University of Paderborn explains that UML (Unified Modeling Language) should be extended to allow development of user interfaces in order to assist end-users to program -

"The UML is strong in modelling the internal functionality of a software system, but weak in modelling user interface aspects of a system. Thus, with respect to the well-known MVC (Model-View-Controller) pattern, the UML supports the Model aspect, while neglecting the View and Controller aspect."

In 'Position Paper for Dagstuhl 2007 EUSE Workshop' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1094 Mary Beth Rosson of Pennsylvania State University explains their development of a drag and drop interface for web based visual programming.

In 'Rethinking the Software Life Cycle: About the Interlace of Different Design and Development Activities' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1084 Yvonne Dittrich of University of Copenhagen argues that more research is needed into the software lifecycle and methods and tools needed for end-user developers, especially when they are collaborating. These end-users often need to adjust old software for new purposes.

In 'Software environments for supporting End-User Development' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1081 Maria Francesca Costabile, Antonio Piccinno also explain that new methodologies and environments are required for supporting end-user collaborative development.

In 'What is an End-User Software Engineer?' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1080 Steven Clarke of Microsoft examines the characteristics of end-user developers in order to assist with meeting their needs.

My home page is http://www.cems.uwe.ac.uk/~phale/.

I have a page on End-User Programming at http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm.

Dagstuhl Seminar End-User Software Engineering - Part 3

End-User Software Engineering - Dagstuhl Seminar - Summary - http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=2007081 - PDF Abstracts and links to papers - http://eusesconsortium.org/docs/dagstuhl_2007.pdf - Margaret M. Burnett, Gregor Engels, Brad A. Myers and Gregg Rothermel - From 18.01.07 to 23.02.07

In 'Exploiting Domain-Specific Structures For End-User Programming Support Tools' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1086 Robin Abraham and Martin Erwig of Oregon State University integrate spreadsheet modelling into the UML modelling process.

In 'Interdisciplinary Design Research for End-User Software Engineering' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1078 Alan Blackwell explains the need for interdisciplinary research on the end-user programming problem to identify techniques within software engineering that can assist with this problem.

In 'Meta-Design: A Conceptual Framework for End-User Software Engineering' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1087 Gerhard Fischer of University of Colorado, Center for LifeLong Learning and Design (L3D) explains the concept of Meta-Design -

"Meta-design is an emerging conceptual framework aimed at defining and creating social and technical infrastructures in which new forms of collaborative design can take place. It extends the traditional notion of system design beyond the original development of a system. It is grounded in the basic assumption that future uses and problems cannot be completely anticipated at design time, when a system is developed."

Fischer also explains that it is the mismatches between their needs and the software support that enables new understandings.

Fischer argues the need for allowing end-users to design software -

"A great amount of new media is designed to see humans only as consumers. The importance of meta-design rests on the fundamental belief that humans (not all of them, not at all times, not in all contexts) want to be and act as designers in personally meaningful activities. Meta-design encourages users to be actively engaged in generating creative extensions to the artifacts given to them and has the potential to break down the strict counterproductive barriers between consumers and designers."

Fischer also argues that software development can never be completely delegated to software professionals because domain experts are the only people that fully understand the domain specific tasks that must be performed.

He puts forward a hypothesis that this emphasis on end-user development also changes the emphasis on testing -

"Software testing is conducted differently. Because domain expert developers themselves are the primary users, complete testing is not as important as in the case when the developers are not the users."

He also argues for an approach to enabling end-user programming possible that makes it interesting to end-users -

"Many times the problem is not that programming is difficult, but that it is boring (as we were told by an artist). Highly creative owners of problems struggle and learn tools that are useful to them, rather than believing in the alternative of "ease-of-use," which limits them to preprogrammed features."

My home page is http://www.cems.uwe.ac.uk/~phale/.

I have a page on End-User Programming at http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm.

Dagstuhl Seminar End-User Software Engineering - Part 2

End-User Software Engineering - Dagstuhl Seminar - Summary - http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=2007081 - PDF Abstracts and links to papers - http://eusesconsortium.org/docs/dagstuhl_2007.pdf - Margaret M. Burnett, Gregor Engels, Brad A. Myers and Gregg Rothermel - From 18.01.07 to 23.02.07, the Dagstuhl Seminar 07081 End-User Software Engineering was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed.

I have been reading this abstracts paper from the Dagstuhl Seminar End-User Software Engineering and it has lots of interesting points to make.

In 'End-User Design' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1099 Professor Alexander Repenning of University of Lugano explains the need for enhancements to UML (Unified Modeling Language) to aid end-user programming.

He makes the point that

"Visual programming languages using drag and drop mechanisms as programming approach make it virtually impossible to create syntactic errors."

So -

"With the syntactic challenge being – more or less – out of the way we can focus on the semantic level of end-user programming."

In 'End-User Development Techniques for Enterprise Resource Planning Software Systems' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1097 Michael Spahn, Stefan Scheidl, and Todor Stoitsev explain SAP research into End-User Programming, they explain -

"End-users of ERP systems are domain experts but not necessarily IT professionals, limiting their ability to adapt the software by themselves to their own needs and forcing them to indirectly influence the adaptation processes by communicating their needs to IT professionals. Empowering the end-users to adapt the software by themselves is an important step in reducing customization costs and enabling high-quality tailoring of software and working environments to the needs of modern information and knowledge workers."

They explain that users must be empowered to explore, process, and analyse information in a user friendly way, and create and adapt reports and queries. An abstraction layer should ease this by hiding technical details to allow for concentrating on business needs.

In 'End-User Software Engineering and Professional End-User Developers' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1095 Judith Segal of the Open University examines issues and problems in Professional end-user development, the author explains -

"These developers are distinguished from other end-user developers in two ways. The first is that, consistent with their being familiar with formal notations and logical scientific reasoning, they tend to have few problems with coding per se. The second is that, as a class, they have a history of developing their own software which long predates the advent of the PC."

In 'End-User Software Engineering Position Paper' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1092 by Henry Lieberman of MIT Media Laboratory he asks -

"Why is it so much harder to program a computer than simply to use a computer application? I can't think of any good reason why this is so; we just happen to have a tradition of arcane programming languages and mystically complex software development techniques. We can do much better."

He goes on to explain -

"HCI has amassed an enormous body of knowledge about what makes interfaces easy to use, and this has been applied widely to many computer applications for end users. Oddly, little of this has been applied to making interfaces for programming easier to use. Non-experts tend to believe that programmers practice a kind of voodoo, perceived to be naturally arcane and mysterious. Since they can handle it so well, programmers aren't perceived as needing ease of use. But we all pay the price for this misconception."

Lieberman explains the goal of this research -

"Finally, as much as is possible, we should make this process as automatic as we can, though the use of program transformation, dependency maintenance, automated reasoning, mixed-initiative interfaces, visualization, and machine learning. Otherwise, I think it will be too much overhead for a non-expert user themselves to keep track of the myriad facets that software development entails. If we succeed in this, people will become End-User Software Engineers without their even realizing it."

My home page is http://www.cems.uwe.ac.uk/~phale/.

I have a page on End-User Programming at http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm.

Dagstuhl Seminar End-User Software Engineering - Part 1

End-User Software Engineering - Dagstuhl Seminar - Summary - http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=2007081 - PDF Abstracts and links to papers - http://eusesconsortium.org/docs/dagstuhl_2007.pdf - Margaret M. Burnett, Gregor Engels, Brad A. Myers and Gregg Rothermel - From 18.01.07 to 23.02.07, the Dagstuhl Seminar 07081 End-User Software Engineering was held in the International Conference and Research Center (IBFI), Schloss Dagstuhl. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed.

I have been reading this abstracts paper from the Dagstuhl Seminar End-User Software Engineering and it has lots of interesting points to make.

In the introduction the report http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1098 states - "The number of end users creating software is far larger than the number of professional programmers. These end users are using various languages and programming systems to create software in forms such as spreadsheets, dynamic web applications, and scientific simulations. This software needs to be sufficiently dependable, but substantial evidence suggests that it is not."

This is a good point, and can be related to a point in 'Barriers to Successful End-User Programming' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1091 by Andrew Ko http://www.cs.cmu.edu/~ajko Carnegie Mellon University about end-user programmers

"Some of them are scientists, some are artists, others are educators and other types of professionals. One thing that all of these people have in common is that their goals are entirely unrelated to producing code."

Andrew Ko explains how this causes a problem -

"the fact that end-user programmers are motivated by their domain and not by the merits of producing high-quality, dependable code, means that most of the barriers that end users encounter in the process of writing a program are perceived as distractions. This is despite the fact that such barriers can represent fundamental problems in end-users' program's or their understanding of how to use a programming language effectively."

the problem is -

"how can tools help end-user programmers' improve their programs' dependability, while allowing them to remain focused on their goals, rather than their code?"

An important part of the solution is to visualise the whole program execution not just the output. Andrew Ko states -

"when a user was trying to diagnose their program's failure, they had to base their hypothesis of what caused the failure just on what they could see in the program's output, rather than on information about the program's execution. In many of these situations, users premature decisions led to errors."

A further problem is that of programs which were intended to be temporary and owned by a particular person becoming central to a company, this often happens with spreadsheets. Ko explains -

"We frequently hear anecdotes about how a one-off excel spreadsheet meant to be temporary became the centerpiece of some accounting logic. How often do such organizational dependencies occur, and how important do such program's become? What can tools do to help the future owners of these programs learn about the program's history and design?"

Mark D Gross explains Carnegie Mellon University work involving end-user programming by designers using diagrams and scratchpads in "Designers Need End User Software Engineering" http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1090.

In "End User Programming for Scientists: Modelling Complex Systems" http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1077 Andrew Begel of Microsoft Research explains -

"Model creation cannot be turned over to a programmer-for-hire without causing the model to become a black box. In order to ensure the validity of the model and stand behind its experimental results, the scientist must be intimately knowledgeable about its innards as well as its outward behaviors."

Text based Computer languages are often too obscure for end-user programmers, Bagel explains -

"Learning a text-based programming language is difficult for novices who want to be programmers. In the first few weeks of learning a language, syntax rules are often the most difficult to comprehend, with semantics interleaved. Non-programmers face these problems, in addition to lacking an engineering mindset to help form mental models of how they want to make the computer do what they want."

In 'End-user (further) development: A case for negotiated semiotic engineering' http://drops.dagstuhl.de/opus/frontdoor.php?source_opus=1083 Clarisse Sieckenius de Souza of Departamento de Informática, PUC-Rio makes the Quote Liberman et al -

"We think that over the next few years, the goal of human-computer interaction (HCI) will evolve from just making system easy to use (even though that goal has not yet been completely achieved) to making systems that are easy to develop"

From

Lieberman, H.; Parternò, F.; Klann, M.; Wulf, V. (2006) End-user development: An emerging paradigm. In End-User Development (Lieberman, Paternò and Wulf, Eds.). Springer. p. 1.

My home page is http://www.cems.uwe.ac.uk/~phale/.

I have a page on End-User Programming at http://www.cems.uwe.ac.uk/amrc/seeds/EndUserProgramming.htm.