It's sometimes pretty difficult to discuss these things in text. When you're face to face with someone you can make all kinds of gestures and drawings to explain how things should work...
[Featured][Alpha] CLAP - TI Link - http://clap.timendus.com
Moderator: MaxCoderz Staff
Glad to have been helpful
It's sometimes pretty difficult to discuss these things in text. When you're face to face with someone you can make all kinds of gestures and drawings to explain how things should work...
It's sometimes pretty difficult to discuss these things in text. When you're face to face with someone you can make all kinds of gestures and drawings to explain how things should work...
http://clap.timendus.com/ - The Calculator Link Alternative Protocol
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
-
KermMartian
- Calc Wizard
- Posts: 549
- Joined: Tue 05 Jul, 2005 11:28 pm
- Contact:
Tell me about it. On that note, have you seen my "LOL Rant"? http://cemetech.net/forum/viewtopic.php?t=969 By the way, whereever it says "0x5", assume it says "lol"; it's a filter my forum has because I got so sick of lol and its misuse.
Also, I thought I'd share a debug tip with you. In order to test my interrupt, I start out by clearing a few bytes on the screen, then, at intervals within the send and receive routines, I do something like: Within the regular code that's running normally, I have Doors CS, so I just hold down the mouse move buttons which causes it to continually ifastcopy. Voila, graphical representation of how far it gets before the collision routines detect an xmit/rec error and ret the interrupt.
Edit: and it only takes 31 clock cycles, so it doesn't cause problems with the actual sending and receiving.
Also, I thought I'd share a debug tip with you. In order to test my interrupt, I start out by clearing a few bytes on the screen, then, at intervals within the send and receive routines, I do something like:
Code: Select all
ld hl,gbuf+12*whatever+whatever
ld a,(hl)
or some bit mask like %01000000
ld (hl),aEdit: and it only takes 31 clock cycles, so it doesn't cause problems with the actual sending and receiving.
Okay, let's get down and dirty with networking 
I will not keep any secrets anymore on how I do things or how I intend to do things. The protocol is mine anyway, even though it's not finished yet, and whoever wants to make an implementation of it is more than welcome. Even if Kerm wishes to make his own implementation for DoorsCS that's compatible with my routines, he's free to do so
I've been thinking about speed and data integrity mainly, which are two big issues in networking about which many, many articles and many books are written. My point of view here is that I want things to work, be simple and preferably be fast. Or at least fast enough to be useful.
I think I'm going to have to make two versions of the library (and the protocol); one that is optimized for simple data transfer between two(!) calculators, mainly aimed at game programmers who need speed and don't care about the theoretical possibilities of networking, and one that is built for networking, showing the possibilities of calc networks and slower applications like turn based strategy games, chatting, file transfers, the lot.
Texas Instruments does not use an RS232 protocol, like I read somewhere and have since believed to be true, according to the information that Wikipedia has on RS232. So I don't know what it uses, but I know how it works, and for some while now I know why they use it.
The TI protocol pulls one line low to send a one or the other to send a zero. The connected calculator pulls the other line low as soon as it has stored the bit to acknowledge it's transfer. I always laughed at that primitive protocol because I expected a clocked protocol to be much faster. Unfortunately my software never really managed to beat the TI protocol...
Because the calculator doesn't have a reliable clock built in, you can't be sure how fast the receiving calculator can process the incoming data. Therefor, to prevent congestion errors, I have to limit the transmission speed to a "safe speed" that empirically has to turn out to be useful. This certainly works, but it is far from optimal. And this is exactely the reason why TI uses their own weird protocol; acknowleding every single bit means that you know exactely when you can send the next bit. So even though it's a silly and slow protocol, there's less time wasted on waiting, and the throughput is dynamic and usually higher than with my own protocol. So basically, that sucks.
Other than that however, the TI protocol has no advantages; it is entirely impossible to use it for networking, for example. That's exactely where my segregation comes into play. I want to make a simple, small and fast library using some implementation of TI's protocol that can exchange bytes, data blocks or strings fast from one calculator to another to accomodate game programmers, and a more complex networking library using my own protocol.
The former is not a very interesting thing to talk about, because it uses known technology in a welltried manner and only the wrapping is more shiny than TI's version (it should be more easy to use, better documented, and perhaps a tad faster, but that's about it). I hope to be able to release that rather soon. If I'm really lazy, I'll even use Ben's routines (with his permission I hope), or the built-in TI routines underneath my shiny wrapping. You'll understand that the keyword here is simplicity and supporting game programmers in coding multiplayer games.
The latter however, will be in perpetual beta as it has been the last couple of months, just to further investigate the possibilities and boundaries of calculator networking as my personal playground. I've been brainstorming a lot about a form of "agreed upon transfer speed" to close the gap with TI's protocol a bit in terms of speed.
The network needs to have a certain speed setting that nodes can, kind of, query the primary calculator about, and a variable containing the address of the "slowest" calculator. You can't in any way store the speed, because different calculators have a different grasp of "time", whether you define time in clockcycles or in milliseconds. This is why the speed has to be measured by each calculator individually from the primary calculator. The slowest calculator has to be known, because when a new calc joins the network it's speed has to be compared to the slowest speed to see if we need to set a new maximum speed.
The maximum possible transfer speed with any calculator can be determined by introducing a bit of error checking on the receiving end. We cannot acknowledge every bit in a network like the TI protocol does because it would interfere, so we have to take acknowledging a level higher. When sending data over, two things can go wrong. Bits can flip because of interference (which increases with higher speeds) or because the receiver can't keep up, or the receiver can miss a few bits after a transfer. These are two things we can check on. Missing bits is an easy one; I can use a timeout after which the transfer gets closed brute force, and I know how many bytes I should receive (preceding length byte), so when my timer fires and I'm missing bits there's something wrong and I can inform the sender that the data was corrupted.
Flipped bits is a whole other topic. Again; books have been written about error detection and correction, but I'll stick to the simplest option, taking the risk of letting a few mistakes pass unnoticed. I'll probably introduce a parity bit with every byte I send, indicating how many (an even or an odd number) bits in the sent byte should have been one. If the parity bit doesn't agree with the data on the receiving end, then there's an odd number of bits flipped. If an even number of bits have flipped though, a parity bit will not be enough. But since I check parity for every byte, and only one byte in a stream has to be corrupt for the transfer to go wrong, I guess that'll solve itself.
So we can detect transfer problems in two ways, and inform either the user (the program that's using the CLAP) or the library about this. I guess I'd best let that depend on what state we're in. If we're testing the connection speed, I want the library to handle the error by sending back a NACK so the primary calculator to throttle the network if neccessary. If we're in normal receive mode I'll inform the user with some switch that we've received data, but that it is corrupted.
Another option would of course be requesting a retransmit from the sender of the stream, to assure correct transfers, but that's starting to look like a reliable data transfer protocol a bit too much to be funny
I'd need to introduce all kinds of buffers and whatnot, and I'm not in the mood for that
So, bottom line is this:
Calculator connects >
Measure maximum speed between slowest nodes by increasing the speed untill it goes wrong >
Adapt network speed to maximum speed >
Transfers should be pretty reliable, if they are corrupt anyway: user gets informed
This shouldn't throttle the network any more than absolutely neccessary, but still make it possible to connect all kinds of processors with different speeds. (This has been a concern of mine from the beginning; I wanted to make a networking library for all calculator models, though I didn't want to statically limit the speed to the slowest calculator type...)
That's about it for now, your ideas and feedback are welcome as always
I will not keep any secrets anymore on how I do things or how I intend to do things. The protocol is mine anyway, even though it's not finished yet, and whoever wants to make an implementation of it is more than welcome. Even if Kerm wishes to make his own implementation for DoorsCS that's compatible with my routines, he's free to do so
I've been thinking about speed and data integrity mainly, which are two big issues in networking about which many, many articles and many books are written. My point of view here is that I want things to work, be simple and preferably be fast. Or at least fast enough to be useful.
I think I'm going to have to make two versions of the library (and the protocol); one that is optimized for simple data transfer between two(!) calculators, mainly aimed at game programmers who need speed and don't care about the theoretical possibilities of networking, and one that is built for networking, showing the possibilities of calc networks and slower applications like turn based strategy games, chatting, file transfers, the lot.
Texas Instruments does not use an RS232 protocol, like I read somewhere and have since believed to be true, according to the information that Wikipedia has on RS232. So I don't know what it uses, but I know how it works, and for some while now I know why they use it.
The TI protocol pulls one line low to send a one or the other to send a zero. The connected calculator pulls the other line low as soon as it has stored the bit to acknowledge it's transfer. I always laughed at that primitive protocol because I expected a clocked protocol to be much faster. Unfortunately my software never really managed to beat the TI protocol...
Because the calculator doesn't have a reliable clock built in, you can't be sure how fast the receiving calculator can process the incoming data. Therefor, to prevent congestion errors, I have to limit the transmission speed to a "safe speed" that empirically has to turn out to be useful. This certainly works, but it is far from optimal. And this is exactely the reason why TI uses their own weird protocol; acknowleding every single bit means that you know exactely when you can send the next bit. So even though it's a silly and slow protocol, there's less time wasted on waiting, and the throughput is dynamic and usually higher than with my own protocol. So basically, that sucks.
Other than that however, the TI protocol has no advantages; it is entirely impossible to use it for networking, for example. That's exactely where my segregation comes into play. I want to make a simple, small and fast library using some implementation of TI's protocol that can exchange bytes, data blocks or strings fast from one calculator to another to accomodate game programmers, and a more complex networking library using my own protocol.
The former is not a very interesting thing to talk about, because it uses known technology in a welltried manner and only the wrapping is more shiny than TI's version (it should be more easy to use, better documented, and perhaps a tad faster, but that's about it). I hope to be able to release that rather soon. If I'm really lazy, I'll even use Ben's routines (with his permission I hope), or the built-in TI routines underneath my shiny wrapping. You'll understand that the keyword here is simplicity and supporting game programmers in coding multiplayer games.
The latter however, will be in perpetual beta as it has been the last couple of months, just to further investigate the possibilities and boundaries of calculator networking as my personal playground. I've been brainstorming a lot about a form of "agreed upon transfer speed" to close the gap with TI's protocol a bit in terms of speed.
The network needs to have a certain speed setting that nodes can, kind of, query the primary calculator about, and a variable containing the address of the "slowest" calculator. You can't in any way store the speed, because different calculators have a different grasp of "time", whether you define time in clockcycles or in milliseconds. This is why the speed has to be measured by each calculator individually from the primary calculator. The slowest calculator has to be known, because when a new calc joins the network it's speed has to be compared to the slowest speed to see if we need to set a new maximum speed.
The maximum possible transfer speed with any calculator can be determined by introducing a bit of error checking on the receiving end. We cannot acknowledge every bit in a network like the TI protocol does because it would interfere, so we have to take acknowledging a level higher. When sending data over, two things can go wrong. Bits can flip because of interference (which increases with higher speeds) or because the receiver can't keep up, or the receiver can miss a few bits after a transfer. These are two things we can check on. Missing bits is an easy one; I can use a timeout after which the transfer gets closed brute force, and I know how many bytes I should receive (preceding length byte), so when my timer fires and I'm missing bits there's something wrong and I can inform the sender that the data was corrupted.
Flipped bits is a whole other topic. Again; books have been written about error detection and correction, but I'll stick to the simplest option, taking the risk of letting a few mistakes pass unnoticed. I'll probably introduce a parity bit with every byte I send, indicating how many (an even or an odd number) bits in the sent byte should have been one. If the parity bit doesn't agree with the data on the receiving end, then there's an odd number of bits flipped. If an even number of bits have flipped though, a parity bit will not be enough. But since I check parity for every byte, and only one byte in a stream has to be corrupt for the transfer to go wrong, I guess that'll solve itself.
So we can detect transfer problems in two ways, and inform either the user (the program that's using the CLAP) or the library about this. I guess I'd best let that depend on what state we're in. If we're testing the connection speed, I want the library to handle the error by sending back a NACK so the primary calculator to throttle the network if neccessary. If we're in normal receive mode I'll inform the user with some switch that we've received data, but that it is corrupted.
Another option would of course be requesting a retransmit from the sender of the stream, to assure correct transfers, but that's starting to look like a reliable data transfer protocol a bit too much to be funny
I'd need to introduce all kinds of buffers and whatnot, and I'm not in the mood for that So, bottom line is this:
Calculator connects >
Measure maximum speed between slowest nodes by increasing the speed untill it goes wrong >
Adapt network speed to maximum speed >
Transfers should be pretty reliable, if they are corrupt anyway: user gets informed
This shouldn't throttle the network any more than absolutely neccessary, but still make it possible to connect all kinds of processors with different speeds. (This has been a concern of mine from the beginning; I wanted to make a networking library for all calculator models, though I didn't want to statically limit the speed to the slowest calculator type...)
That's about it for now, your ideas and feedback are welcome as always
http://clap.timendus.com/ - The Calculator Link Alternative Protocol
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
-
benryves
- Maxcoderz Staff
- Posts: 3080
- Joined: Thu 16 Dec, 2004 10:06 pm
- Location: Croydon, England
- Contact:
Interesting stuff indeed. I always went under the assumption that a clocked protocol is a pain with the TI, what with no accurate timing hardware (until recently, that is).
Feel free to use my routines, by the way.
The AT protocol is fact clocked. (AT is PS/2, the PS/2 just has a different shaped socket). The connected device generates the clock - thankfully it's at a low enough speed for our TIs to keep up with. The protocol is useless to us (it's heavily biased to being a one-way system).
Have you looked into MV's "Tachyon" link? I can't find any mention of it on ticalc.org, but have it on my HDD (along with some rather ancient documents by yourself) if you wish to look at it.
Feel free to use my routines, by the way.
The AT protocol is fact clocked. (AT is PS/2, the PS/2 just has a different shaped socket). The connected device generates the clock - thankfully it's at a low enough speed for our TIs to keep up with. The protocol is useless to us (it's heavily biased to being a one-way system).
Have you looked into MV's "Tachyon" link? I can't find any mention of it on ticalc.org, but have it on my HDD (along with some rather ancient documents by yourself) if you wish to look at it.
-
KermMartian
- Calc Wizard
- Posts: 549
- Joined: Tue 05 Jul, 2005 11:28 pm
- Contact:
Timendus wrote:I'll probably introduce a parity bit with every byte I send, indicating how many (an even or an odd number) bits in the sent byte should have been one. If the parity bit doesn't agree with the data on the receiving end, then there's an odd number of bits flipped. If an even number of bits have flipped though, a parity bit will not be enough. But since I check parity for every byte, and only one byte in a stream has to be corrupt for the transfer to go wrong, I guess that'll solve itself.
(1) Would it be safe to assume that the part about "that'll solve itself" means that you will also calculate a checksum for the transfered data?(2) Also, I believe I may have heard that there is a theoretical limit to either the length of the link cables and/or the number of calculators connected in a "network"... something about it being due to the amount of current being too high, or too low... am I recalling this correctly, and how are you planning to take this into account?
(3) One more thing, are you going to be "requiring" a linking "hub" or are you thinking of simply having each calc use a 2-1 adapter to add a new calculator to the network?
(These were two considerations I had when I was theorizing about the networking possibilities for a 3rd-party calculator OS.)
____________________
Reaktix Software (formally XDG Kat-Productions)
-Michael
MyBlog ChatWithMe
Thanks!CompWiz wrote:If I ever finish my Advance Wars clone(don't hold your breath), I'll be sure to put the multi-calc linking in
It is indeed. Potentially fast though, and pretty much required for linking >2 calculators... Another way to do things would be tip-ring = 1, ring-tip = 0 but that too would have a predefined speed that the receiver has to adapt to.benryves wrote:I always went under the assumption that a clocked protocol is a pain with the TI, what with no accurate timing hardware (until recently, that is).
Thanks! I think I will.Feel free to use my routines, by the way.
With that you mean that the clock is generated by the keyboard and data can be read on the falling edge, right? So I take it Kerm claiming he's using "AT-style fastclocking" - whatever that may be - can safely be ignored?The AT protocol is fact clocked.
I have seen those routines, if I'm not mistaken they use the same protocol as the CLAP currently does. He probably brought the timing further to the edge to make it deserve it's nameHave you looked into MV's "Tachyon" link? I can't find any mention of it on ticalc.org, but have it on my HDD (along with some rather ancient documents by yourself) if you wish to look at it.
Nope. That part means that there's a negligible chance of errors: I haven't seen any bits flipping on hardware yet, even though my link cables are horribly amateuristic, and if one bit in a stream would happen to flip it'll detect it. If two bits in a byte flip, there'll probably be other affected bytes that flag the stream as invalid. So I estimate the chances of data errors at less than one in a million.Saibot84 wrote:Would it be safe to assume that the part about "that'll solve itself" means that you will also calculate a checksum for the transfered data?
Ah, yes... You wouldn't happen to have read my four year old article would you?Also, I believe I may have heard that there is a theoretical limit to either the length of the link cables and/or the number of calculators connected in a "network"... something about it being due to the amount of current being too high, or too low... am I recalling this correctly, and how are you planning to take this into account?
I wrote something about more calculators giving more current, overshading the low signal. That's just not correct, my mistake There are limits to the usefulness of a multitude of calculators in one network though; more calculators mean more data on the lines, making all calculators run slower (processing data). One broadcasting calculator and lots of listeners shouldn't be any problem though.
On the length of the cables; there is a limit to that because of timing issues and distortion. I will not go into that though because I don't expect anyone to use cables that long. It's not like you're going to wire a calc LAN through your house with several hundred metres of cable
Both are possible, and if you scroll back a few pages in this topic you'll see drawings for both settings. You could even use them intertwinedOne more thing, are you going to be "requiring" a linking "hub" or are you thinking of simply having each calc use a 2-1 adapter to add a new calculator to the network?
Edit: Looked them up for you:
http://kvince83.tengun.net/maxboard/vie ... 5163#45163
http://kvince83.tengun.net/maxboard/vie ... 4551#44551
http://clap.timendus.com/ - The Calculator Link Alternative Protocol
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
(looked at the pictures you linked) That's so cool! Lemme just point out that I've seen (and have two) 2 way headphone splitter 
. Maybe we can find a 2-way splitter like that, but for 2.5mm?
<crazy idea>would it completely negate your efforts to speed up the link protocol if you separated the linking into more than one stream?</crazy>
. Maybe we can find a 2-way splitter like that, but for 2.5mm?<crazy idea>would it completely negate your efforts to speed up the link protocol if you separated the linking into more than one stream?</crazy>
-
KermMartian
- Calc Wizard
- Posts: 549
- Joined: Tue 05 Jul, 2005 11:28 pm
- Contact:
-
King Harold
- Calc King
- Posts: 1513
- Joined: Sat 05 Aug, 2006 7:22 am
Hey Tim, did you ever consider making network with 84's? It should be a lot easier.
It's theoretically easy and you could theoretically add infinite calc's to the network. I've tried it with TI BASIC but for some reason calcs don't react to getCalc( commands when they are in a loop, so basically the getCalc( command is useless. But, if someone would make a very good linking-lib which can also send/get through the USB port even if every calc is looping, then it should be a piece of cake. I don't understand how to use the USB port in asm but you probebly do..
Maybe that's an idea for xLIB aswell? (a bit like omnicalc's sendbyte?)
It's theoretically easy and you could theoretically add infinite calc's to the network. I've tried it with TI BASIC but for some reason calcs don't react to getCalc( commands when they are in a loop, so basically the getCalc( command is useless. But, if someone would make a very good linking-lib which can also send/get through the USB port even if every calc is looping, then it should be a piece of cake. I don't understand how to use the USB port in asm but you probebly do..
Maybe that's an idea for xLIB aswell? (a bit like omnicalc's sendbyte?)
Maybe you can. I'm guessing it would work pretty well, if it fits in the link port. Anyway, it's pretty easy to adapt a link cable yourself, or make a simple hub, so it shouldn't be a problem anyway.Saibot84 wrote:Maybe we can find a 2-way splitter like that, but for 2.5mm?
How exactely did you have that in mind..?<crazy idea>would it completely negate your efforts to speed up the link protocol if you separated the linking into more than one stream?</crazy>
Yes, that's what I thought you meant. So it's identical to the CLAP in it's current state in that aspect.By "AT-style fastclocking I meant that the sender determines transfer rate and assumes that the receiver can keep up.
Heh, no wayHey Tim, did you ever consider making network with 84's? It should be a lot easier.
First; I don't have one, so I can't test with it. Second; I don't know anything about the USB port. Third; It is not in any way easier to chain calculators together through two different protocols. You'd need to write two different "drivers" as opposed to one, you'd need to implement some kind of routing to be able to get messages across the network, et cetera...
Thanks for the input and ideas everyone!
http://clap.timendus.com/ - The Calculator Link Alternative Protocol
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
http://api.timendus.com/ - Make your life easier, leave the coding to the API
http://vera.timendus.com/ - The calc lover's OS
-
KermMartian
- Calc Wizard
- Posts: 549
- Joined: Tue 05 Jul, 2005 11:28 pm
- Contact:
what I had in mind was something to the effect of:
we have 4 calcs hooked up together... calc 1 needs to "talk" to calc 2... while they are talking, calc 3 needs to "talk" to calc 4... so maybe calc 1 and 3 alternate sending bits while calcs 2 and 4 alternate receiving bits over the link... this could also be used to have more than just 2 conversations btwn calcs. but it will definitely slow down the linking.
we have 4 calcs hooked up together... calc 1 needs to "talk" to calc 2... while they are talking, calc 3 needs to "talk" to calc 4... so maybe calc 1 and 3 alternate sending bits while calcs 2 and 4 alternate receiving bits over the link... this could also be used to have more than just 2 conversations btwn calcs. but it will definitely slow down the linking.