Java Virtual Machine

[Timendus]

I knew that from Ben

[Kalimero]

I'm a Belgian too.

There is another thing technology won't change, and that would be the demand for creating products of high standards. We have the necessary means to create quality software in our disposition for decades. It's all a matter of organisational skills to use them properly.

Same with society. We have the necessary means to feed the entire planet. All it would need is organisation, but that's not going to happen. To explain that in mathematical terms (and thereby staying somewhat on topic :), the system of differential equations that describe our society don't allow that. It would take to much effort. If we want to feed the entire planet we need to change our set of diff eqs so that they will lead us to a better equilibrium without much effort. Of course I don't know what changes, but I'm sure with current laws it won't work, no matter how hard some people try. All (I think) I know is that we need fundamental changes.

Back to software :) If we want better software, we have to stop focussing on producing code and spend more time designing. The academic world knows this and they teach it to every programmer, but they don't give us the proper tools. That makes designing time consuming and boring, much like commenting code, writing documentation etc. What's far worse though is that if you want to change your design you need to refactor your entire codebase. Few people do that. Most just patch and patch until the code becomes a cancer, starts mutating and grows in all possible directions.

I've been thinking on how such a design tool would look like, but nothing in detail. The basic idea is that you need to be able to split up a project into subprojects or subparts or sublayers. Then you can zoom in on such subpart and split it up even further and so on. Each such part provides a service to its parent. The parent combines the services from the subparts to provide an even more advanced service to its own parent.

If there's some way to describe the service a part provides (API?), then you can use it in your design. You can always implement it with more detail later. Also, refactoring becomes easier because parts are completely independent of each other. You can disable them, reimplement them, reuse them somewhere else, etc.

A part can be pretty much anything. Suppose you want to make a GUI for a database, then you'd combine a database part with some GUI parts. A while loop can be a part too. And it can combine three if parts for example.

Anyway, so far the basic idea I have in mind. There are still a lot of missing details I know. Sadly I don't have the time to work on this some more.

[CoBB]

But programs aren't tree-like, not even dependencies. Getting down to the level of control primitives in the visual design isn't as good as it sounds either, have you ever been cursed to work with LabVIEW?

There's also a danger of premature abstraction. It was noted in other areas that visually drawn structures are much less effective than those coded in text, because the latter enforces better structuring psychologically. Probably it was about designing ontologies, I'm not sure. Anyway, bad design can easily introduce unexpected computational complexity into apparently simple processes. If the design is enforced, you just shot yourself in the foot, because you distributed the problem on different levels which is very hard to visualise and fix. If it isn't enforced, people will just go ahead and break it, and we are back to present practices again.

How will these tools protect you from making bad design? That's the principal question. If they don't even try to, we haven't gained anything.

[Kalimero]

But programs aren't tree-like, not even dependencies.

Says who? To me software is like a black box or a magic box if you will. It takes input, does magical things and produces output. You don't need to know what's on the inside of that box. All you care about are its characteristics. You can then combine several of these boxes to create a bigger magic box that takes some input, does its magical stuff using the smaller magical boxes and produces some output. This big box doesn't care about how these tiny boxes do their job and these tiny boxes don't care about what the other tiny boxes do. They all just magically and happily do their job. And this applies to pretty much any engineering task. You combine things to make something new.
And the beauty of this analogy is that any programming style/paradigm can be derived from it. Procedural, imperative, declarative, functional, object oriented even with multiple inheritence, language oriented,... you name it. They are all a specific way to combine magical boxes.

Getting down to the level of control primitives in the visual design isn't as good as it sounds either, have you ever been cursed to work with LabVIEW?

Can't say I have, but I've taken a look at its website and I can imagine why you call it a curse. However, visual design like that is not really what I have in mind. I was thinking about something with a lot more structure.

There's also a danger of premature abstraction. It was noted in other areas that visually drawn structures are much less effective than those coded in text, because the latter enforces better structuring psychologically.

I can follow that, at least for software. I can't imaging drawing a house in text mode though. Anyway, I was thinking of representing a magic box by a table or some sort of container. Such a table or container would define some cells or fields in which in you store other containers. A while-container for example would have a field to store the expression to evaluate, in other words a container that returns a boolean value. The body of a while-container would allow for multiple containers to be stored. If you want to make it a bit more advanced you can display such containers using some sort of stylesheet defined by the programmer. Some may prefer while...do...done instead of curly brackets... or a background color... it's all possible. I still need to find a place to store preconditions, postconditions and any tests you want run. Maybe they can also be put in some field...

Anyway, bad design can easily introduce unexpected computational complexity into apparently simple processes. If the design is enforced, you just shot yourself in the foot, because you distributed the problem on different levels which is very hard to visualise and fix. If it isn't enforced, people will just go ahead and break it, and we are back to present practices again.

I can follow you on the complexity thing, but I don't understand what you mean with the design being enforced or not. A magic box does what it's supposed to do, and can only do that and nothing else. Its design is implicitly enforced that way.

How will these tools protect you from making bad design? That's the principal question. If they don't even try to, we haven't gained anything.

Well today I don't know any tool that allows me to make/verify/change a software design in an efficient/elegant way. Though Eiffel comes close, pretty much all I can use right now is my head and pencil and paper. The latter quickly becomes a mess and I don't know about your head, but mine sure can't keep a detailed overview of an entire software project, let alone making and verifying a design. Really all I'm looking for is a tool to help me with that and preferably general enough to be applicable to any software project. This is what I could come up with. Maybe you have other ideas and in that case I sure want to hear about them. Where are all those other (future) cs students on this board by the way?

As for efficiency, that's what takes skill and education. If I were to design a building without a design tool, I'm sure it will collapse (aka crash, it has bugs). If I do have a design tool that calculates all the forces for me, then it may not collapse, but it'll likely be very ineffecient in use of resources. That's because I'm not educated as an architect. Now apply the same to software architecture and imagine a world where there are no bugs. And with that I mean what you could call runtime bugs, because a particular feature can still be implemented in the wrong way, which could also be called a bug. That applies to any engineering branch though.

[Patori]

*sleeps, ignore debate*

[Kalimero]

Hehe, maybe you're to young to understand. Thing is, large software projects are difficult manage. Ask Microsoft why Longhorn takes so long.

[Timendus]

Where are all those other (future) cs students on this board by the way?

Oh, I'm sorry... You were expecting us to join the discussion..? ;)

Well... I would agree that a tool that detects runtime bugs would be very useful. But I haven't yet seen a complex programming tool that isn't a total pain in the ass most of the time, no matter how right it is So I'd expect such a tool to be used incorrectly, to be ignored, et cetera... But it could help a bit...

We (as in; us humans) haven't been programming for very long yet. We've been designing buildings for thousands of years, and we can now profit from the knowledge we've collected in those years. Designing software is something totally new that we've only just begon with, and I think we have to accept to a certain extent that our "buildings" will "collapse" from time to time. Especially the large ones
(That's why I find it hard to understand how people can often blindly trust computersystems...)
And that also implies that we do indeed have to design our "buildings" like the Egyptians did: with pencil and paper.

And I agree with you that designing software is (or should be) combining magic boxes. It makes the process a lot less complicated. When I design something, I start by drawing a visual representation of the large boxes, and then I start programming, and I "describe" what the large box does like
doThis();
doThat(getSomething());
And last I code those little boxes. I suppose most people do that.

[CoBB]

Says who? To me software is like a black box or a magic box if you will.

Certainly, but I was talking about the relations between these black boxes. E. g. if you want to create a box that does FFT, you'll need one that calculates the sine function. But this function isn't exclusive to this application, many others will also use this sine box--that's why I said the dependency graph isn't a tree. The hard part is not implementing the boxes but breaking up any given problem into boxes, i. e. determining the borders. No-one is preventing you from coming up with a decomposition which results in circular dependency or just some serious and untraceable drop in performance (insidious design errors cannot be tracked by profiling). Simply postulating the existence of the concept of black box abstraction is unfortunately non-constructive...

If you want to make it a bit more advanced you can display such containers using some sort of stylesheet defined by the programmer. Some may prefer while...do...done instead of curly brackets... or a background color... it's all possible. I still need to find a place to store preconditions, postconditions and any tests you want run. Maybe they can also be put in some field...

Basically you are talking about a metalanguage here which allows you to sort of customise the grammar of the source. But I don't see a principal difference from what we have today. It seems like an unnecessary burden to introduce a new language layer which essentially doesn't add new functionality.

I can follow you on the complexity thing, but I don't understand what you mean with the design being enforced or not. A magic box does what it's supposed to do, and can only do that and nothing else. Its design is implicitly enforced that way.

Enforced meaning that the machine generated code cannot be altered manually.

Well today I don't know any tool that allows me to make/verify/change a software design in an efficient/elegant way.

I believe it's unlikely to happen any soon, see below.

This is what I could come up with. Maybe you have other ideas and in that case I sure want to hear about them. Where are all those other (future) cs students on this board by the way?

I have two short stories here.

There is a nice tool called LPTP that can verify the 'correctness' of pure Prolog programs. In practice that means you can formulate theorems concerning the desired behaviour of the program and try to prove them. It comes with pre-done proofs concerning some useful stuff. The proofs that assure you about the correctness of the three basic operations on integers (+, - and *) take up more than 3000 lines altogether, although they are based on Peano algebra which cuts down on their size a lot. I tried to prove that multiplication can be done faster than repeated addition (using shifts) and yield the same result, and I couldn't do it after being immersed for days, although the program was trivial, a five-liner, and you could tell it's doing what it's supposed to by just looking at it.

There is another 'nice' little verification tool called SPIN. It lets you analyse the behaviour of parallel processes written in a language called Promela. Correctness is ensured through assertions. Unfortunately, the only way to solve this problem is an exhaustive search in the space of program states. You can guess what happened: another trivially correct (one-page) program had such a huge number of different states that the system choked on it immediately. (It ran out of memory in a depth-first search after reaching about the 12 millionth level.) It doesn't mean that this tool is completely useless, only it can work with oversimplified models of processes and protocols.

In the case of design we can't even tell what 'correctness' means. At least in the above cases we had a formal definition and a working implementation. (As you know, the scientific scene doesn't respect working implementations, only publications... No wonder I found lots of ideas in the topic of formal verification that only exist on paper.) Analysis has its limitations anyway, just think of the halting problem or the decidability problems that come up in logic. Which means, all in all, that even if we could come up with a sensible formalism to handle design related questions (probably based on some modal logic), we couldn't do anything practically useful with it. Well, you can always make it a thesis.

Now apply the same to software architecture and imagine a world where there are no bugs. And with that I mean what you could call runtime bugs, because a particular feature can still be implemented in the wrong way, which could also be called a bug. That applies to any engineering branch though.

Design cannot solve problems on algorithmic level, which is responsible for what you call runtime bugs. This is where declarative languages could help a lot, but they are not really appropriate when you have to work with stateful entities.

Sir Hoare was in Hungary recently and gave an hour presentation at our university about verifying compilers. Interesting stuff. I think that's much more relevant to bugs, I wouldn't count on improving design practices to solve these kinds of problems.