Bot Thoughts

[iago]

So I'm tossing around the idea of writing something new for Battle.net, from scratch, using some of the stuff I've learned more recently.

What I'd like to do is write a server that would allow people to hit it over a standard protocol (I'm thinking SOAP or something similar). It would probably be written in Java, but not guaranteed. It would have different services, including CheckRevision and the like, and additionally have bot functionality (create a new bot instance, connect to Battle.net, etc.).

A typical authenticated session would be:
- Client program hits the authentication service, supplies username/password (or certificate, if I go that way)
- Server sends back a token (sort of like a kerberos token, but maybe simpler, don't know)
- Clients makes requests, sending the token with each one
- Token expires after a preset time, 300s or something

I'd document all the public services, from cdkey hashing to version checking to viewing/using bots. Authorization would be a mix of discretionary access control and role-based access control.

It would, of course, be 100% opensource.

Here's one catch: I would NOT write a client program. But I would publish, document, and test the public interface, and would use a standard protocol (like I said, I'm thinking SOAP). But I hate writing GUIs. I would need somebody else to volunteer to write one, whether it's in Java, C++, Python, C#, VB, or whatever, I don't care.

Right now, the idea is just in my mind. If I decide to go ahead, it might be a summer project for me. I'm just reaching out for some ideas, though. Any suggestions, brainstorming, or whatever is encouraged. Also, the technologies I use are wide open, let me know what you'd prefer to see and I'll consider it.

[Blaze]

I like the idea, and I'd definitely write a client for that. 

[MyndFyre]

Oh god, SOAP?  For something that was once binary that's totally, totally lame.

I'd written specs for (what I called) RSP: http://www.jinxbot.net/rsp/ - it's a similar thought and similar authentication, supporting filtering protocols as well.

But - all told - I don't have time to implement an RSP server anymore.  Might be interesting to look at in design, though.

[iago]

Well, I definitely don't want to use a binary protocol. There is no compelling reason to use a binary protocol these days, there are far more negatives than positives. Especially since I want to go cross-platform and implementation-independent -- binary protocols will complicate things far more than it's worth.

I think the way to go is to build it on top of an open protocol designed for this kind of thing. It makes the communication part far easier and will make implementation far faster in any language. I think that's such a huge positive that it's not worth considering making yet another proprietary binary protocol.

That being said, I know very little about SOAP. I have no compelling reason to use it, really. If anybody can make a suggestion for a better protocol (XML is preferred, but others are fine), I'm happy to consider it. It's not like I'm going to be implementing this in the next couple months, or possibly ever, I just want to consider things.

[zorm]

Generally my experience has been that binary protocols are infact better for this type of thing. Specifying which endian is used and how big everything is in bits is sooooo much easier then worrying about things like "what happens if there is extra whitespace? Do things explode?"

Not to mention that most languages have nativish support for things like switching endians and processing binary data. When you get into XML land everyone has to go out and get a 3rd party library thats probably huge and sluggish in order to process that data.

[iago]

I have to completely disagree.

One huge disadvantage to a binary protocol is that every packet has to be specified up front. If you change a packet with a new version, you have to either break compatibility with old clients or specify a whole new packet. With an XML protocol, for example, a new field or attribute can be added without breaking old clients. Plus, I can easily go back and add things that I forgot. In fact, if I understand SOAP properly, in most languages I can just change the definition of the object that's sent, and it automatically sends the new object without any other intervention.

With an XML protocol, I can also layer things. For example, if I want to let somebody make separate connections to the server, they can choose to add an authentication section to the packet with their token. That way, they will be authorized over multiple sessions simply by including a token.

With binary packets, you have to worry about data types. If you define something as a byte, short, or int, you're limited to 2⁸, 2¹⁶, or 2³² distinct values. On the other hand, if you use an ascii protocol, you can define a field as a generic number, and it can be read in in whichever format the language supports.

Also, it's not "sooooo" much easier to fiddle around with bits than to use a well-defined protocol. XML, for example, is defined quite clearly, and every language has good XML-processing libraries. I don't have to worry about whitespace or anything like that, I can create a specification that everybody who reads it understands how to follow. If I decide to go with SOAP (or another well-defined protocol), it still has strong language support and most of the marshaling is looked after without any intervention.

Another disadvantage to binary protocols is that somebody can't just look at it and understand it. They have to get a packet definition and a hex editor and start counting bytes. With an XML protocol, you can just look at the packet and you know exactly what's going on (assuming the programmer didn't do something stupid). This readability makes debugging and understanding much easier.

Those are the best reasons I can think of to use a non-binary protocol. I like the idea of XML and SOAP, although Web services also interest me. I don't know a whole lot about any of those, though, so it'd be an interesting time to do some research.

[MyndFyre]

iago, honestly, that's not been my experience with XML, but whatever.

In addition, you'll notice my binary protocol supports layering. 

As for SOAP, most languages that support mapping tools and auto-generated WSDL/schemas-to-code will crap on you when you change the SOAP spec.  At least that's how things went when I was working in Flash remoting with SOAP.

As for the binary packets, you're not limited - if worst comes to worst you can specify alternative methods of storing values, if you know that you'll need more than 18,446,744,073,709,551,615 different values, then you can always write out the value in ASCII.  BUT - if you're going to do something meaningful with the numeric data, chances are you'll have to bring it back to a binary representation anyway.

IMO the only real point you've got here is the readability. 

If you're determined to use an XML based protocol, I'd suggest Rest or XML-RPC.

[Camel]

In fact, if I understand SOAP properly, in most languages I can just change the definition of the object that's sent, and it automatically sends the new object without any other intervention.

Pretty much.

A SOAP packet is the manifestation of an instance of a Message. I use CXF at work, where the message is a Java class which defines the attributes and fields that may exist inside of the SOAP message, and it marks these as such using annotations (those annoying full-liners that are prefixed by the '@' character; they're sort of like Java's view of what preprocessor directives should be). That means in order to add an attribute, you'd only need to add two lines of code to your Message class:

Code: [Select]

@XmlRootElement(name = "someResponse", namespace = "http://mynamespace.com")
@XmlType(name = "SomeResponse")
public class SomeResponse {
       
        @XmlAttribute(name = "my-attr")
        public String myAttr = "myAttryValue";
       
}

If you're determined to use an XML based protocol, I'd suggest Rest or XML-RPC.

For what purpose? Why are REST or XML-RPC any better?

Here's the bottom line: they're all identical. They're exactly the same in all aspects:
They're all simple
They're all built on XML
They're all easy to understand

So, what's the advantage? SOAP is the industry standard. Because of that, it's more widely used, there are more libraries available for it, and your compatibility quotient will go up. To me, this sounds like exactly what iago wants.

[iago]

So besides XML, there's also Web services. What can you guys tell me about Web services? Are they generic, useful, and supported across different languages?

[MyndFyre]

So, what's the advantage? SOAP is the industry standard. Because of that, it's more widely used, there are more libraries available for it, and your compatibility quotient will go up. To me, this sounds like exactly what iago wants.

SOAP is ugly and verbose; it's also difficult to work with in ActionScript, Flex, and JavaScript.  It's FAR easier to work with Rest and XML-RPC in these languages than SOAP.

I think SOAP is the industry standard because Microsoft's code-generation tools for web services hit first.  Now even Microsoft is singing the Rest song over at their presentations.

What does the standard mean, anyway?  Facebook uses Rest.  Google API uses Rest.  Blogger uses XML-RPC.  What's your point?

[iago]

I have no problem with using Rest or XML-RPC. I've never even heard of them, to be honest. I just thought SOAP first because I've heard of it and it seems to have the functionality that I need.

If I actually write this, I'll consider them all. What I want to avoid is a custom protocol or a binary protocol.

[Chavo]

If you are wanting to write something new for Battle.net, I'm not sure that anything login-related is the way to go.  Apart from lockdown, there isn't anything new or needed in that arena.

I don't keep up with other games, but in the Warcraft 3 part of town, battle.net-less interbot communication and the ability to host/join games are easily the most in demand right now.  I'm sure an elegant solution to one or both of those would be very popular (hell, I'd probably write a client..).

[iago]

Well, I'd expose any service I could. Once I got the basic functionality down, I could add new services without breaking old clients. I could also add a way to get a list of possible actions.

[Camel]

SOAP is ugly and verbose; it's also difficult to work with in ActionScript, Flex, and JavaScript.  It's FAR easier to work with Rest and XML-RPC in these languages than SOAP.

A SOAP packet that does not represent an error has two XML tags surrounding the pure user-defined XML. What about that is ugly or verbose?

So besides XML, there's also Web services. What can you guys tell me about Web services? Are they generic, useful, and supported across different languages?

A "Web Service," strictly speaking, simply means an HTTP server that provides information to a client in an indirect way - that is to say, it doesn't simply generate a flat HTML file for a web browser to read. The definition is loose because the different flavors of web services are implemented so differently. A better way to understand what it is and what it does would be to study a specific implementation. Since I'm most familiar with SOAP, I'll explain SOAP-WS.

Per my previous post, you can gather that a SOAP-WS is an HTTP server that provides a SOAP "service" -- a server-side application that responds to queries from a client by exchanging SOAP packets over an HTTP connection.

The format of the SOAP packet is defined by the WSDL document - the Web Service Description Language is what's used to define types, messages, operations, faults, ports, bindings, and services.

Italicized items are SOAP concepts; underlined items are Java concepts.

Types: Things that are provided by the SOAP spec include 'int' and 'string' - you can create complex types such as lists

Messages: A special Type that is used for Operation input, output, and faults. An instance of a Message manifests as a SOAP packet.

Operations: A method that the Service advertises.

Faults: An exception that the Operation throws.

Ports: An interface that the Service's class implements. It defines a set of Operations that the Service will advertise.

Binding: The relationship between a Port and a Service. The Binding defines the language which the Service will communicate in (ie, SOAP).

Service: A an instance of the implementation class for a Port.

[warz]

don't do it!

[MyndFyre]

A SOAP packet that does not represent an error has two XML tags surrounding the pure user-defined XML. What about that is ugly or verbose?

Dude, even Wikipedia knows how verbose it is, and I totally didn't write that.

Example:
http://www.clan-aoa.org/aoamod4_login.asmx?op=Login

Two things make this yucky to work with:
1.) Namespacing XML is otherworldly. 
2.) Why bother with an envelope, which really provides no value, when your XML can be RESTful, such as:

Code: [Select]

<Login>
  <Username>string</Username>
  <Password>string</Password>
</Login>


[Camel]

1.) Namespacing XML is otherworldly.

I disagree. When your language is weakly typed, you're living in the past.

2.) Why bother with an envelope, which really provides no value, when your XML can be RESTful, such as:

Code: [Select]

<Login>
  <Username>string</Username>
  <Password>string</Password>
</Login>


The SOAP envelope is the single ancestor; it takes the place of your above "login" element. XML, strictly speaking, requires a single ancestor, so the argument is moot.

The purpose of the SOAP:body tag is to distinguish successful operation result message types from faults. I don't see how you can possibly argue that there's no value to it, when it's pretty clear what the value is just by looking at any example.

[Skywing]

If you use a TLVs (a-la RADIUS), binary protocols easily become as extendible as text-based protocols in terms of compatibility.

[Camel]

Someone published a spec for a "binary XML" using TLVs.

[MyndFyre]

iago, this made me think of you

[iago]

I read that this morning.. but why, exactly? Is every use of XML as a Web service a wrong use?

[MyndFyre]

No.   But sometimes, yes. 

[Camel]

I lol'd

[Blaze]

Haha, that's awesome.