Clan x86
Technical (Development, Security, etc.) => General Programming => Botdev => Topic started by: Camel on December 02, 2008, 03:00:20 pm
-
Things I'm particularly interested in include statstrings, which namespace the accounts will belong to, and login procedure (presumably NLS?). Has anyone heard any good rumors, or disassembled any interesting binaries?
From what I've heard, BNCS is going to be seeing a huge amount of new stuff for SC2.
-
Well lets jsut assume they are gunna use NLS v3.
When ti comes to statstrings, I'm going the whole WC3 style for SC and D2 style for D3.
I dont feel like looking, anyone got a firm release date for SC?
-
i speculate everything is doing to use an xml-based protocol. it'll all use udp, too. udp without tcp-mimicing features. it'll use another modified SHA-1.
this is pure speculation, though.
-
Using XML would be pure stupidity.
-
Oh, I forgot to mention that I also speculate that it will bring back channel warring.
-
Using XML would be pure stupidity.
Explain.
-
Using XML would be pure stupidity.
Explain.
It would violate one of the primary laws of physics: blizzard makes no effort to support the botdev community.
-
Using XML would be pure stupidity.
Explain.
It would violate one of the primary laws of physics: blizzard makes no effort to support the botdev community.
That doesn't make using XML stupidity, it makes it unlikely. :P
-
Using XML would be pure stupidity.
Explain.
It would violate one of the primary laws of physics: blizzard makes no effort to support the botdev community.
That doesn't make using XML stupidity, it makes it unlikely. :P
Come on iago. If a game action required a single byte of data, XML would require it to be wrapped in 30-40 bytes of text. It's a waste.
-
Could you even do XML over UDP? Doesn't UDP not guarantee that order is preserved?
-
Come on iago. If a game action required a single byte of data, XML would require it to be wrapped in 30-40 bytes of text. It's a waste.
We're talking about chatting, not about games.
They already send a whackload of metadata with one byte of data. With XML, it would be more expandable, so when they decide to add new features in the future they wouldn't have to hack on a new packet or pad an empty field with 0xbaadf00d anymore. Right now, they're stuck with the identical packets they used 10 years ago.
-
We're talking about chatting, not about games.
Actually, looking back at the thread, that isn't entirely clear. If we're talking about chatting, XML makes sense. If we're talking about gaming, it does not (because the game protocols are program-specific, and can be kept in sync with the server more easily).
-
Come on iago. If a game action required a single byte of data, XML would require it to be wrapped in 30-40 bytes of text. It's a waste.
We're talking about chatting, not about games.
They already send a whackload of metadata with one byte of data. With XML, it would be more expandable, so when they decide to add new features in the future they wouldn't have to hack on a new packet or pad an empty field with 0xbaadf00d anymore. Right now, they're stuck with the identical packets they used 10 years ago.
I speculate that over the next short while, all SID_AUTH system games (everything except for Diablo 1?) will download their Battle.net libraries over BNFTP as part of the logon sequence. That way, only SID_AUTH_INFO, SID_AUTH_CHECK, and BNFTP_* need to remain constant, and Blizzard can fuck up anything else they please.
EDIT -
And as I explained in the other thread, I feel that SID_AUTH_CHECK was designed pretty well and should have no trouble supporting either new game.
-
Come on iago. If a game action required a single byte of data, XML would require it to be wrapped in 30-40 bytes of text. It's a waste.
We're talking about chatting, not about games.
They already send a whackload of metadata with one byte of data. With XML, it would be more expandable, so when they decide to add new features in the future they wouldn't have to hack on a new packet or pad an empty field with 0xbaadf00d anymore. Right now, they're stuck with the identical packets they used 10 years ago.
Eh, I don't know that that's true. Depending on the structure of the client handler, the server could potentially send more data than the client handler expects. For example, supposing they send a packet with header length of 40, but only 25 bytes is used (by the version defined by older clients). Newer clients can recognize that the actual length of the packet is longer and know to take some action with the rest of the data. Nothing has to be done with special markers or anything like that.
-
Didn't the botdev community decide that they most likely only bothered to switch to NLS because of some obscure security flaw -- something that the SID_AUTH_*PROOF packets were supposed to take care of? I don't even remember what information they prove, but I am correct in my assumption, then it seems rather unlikely that they will re-reinvent the login sequence, as far as the packets that are used. We may well see a new version of NLS, though.
[edit] Am I correct in inferring that the client silently discards the unused portion of the packet buffer? I tend to verify that my buffers are empty post-processing for debugging purposes (raise a warning, but continue anyways) for most of my packet handlers.
-
Eh, I don't know that that's true. Depending on the structure of the client handler, the server could potentially send more data than the client handler expects. For example, supposing they send a packet with header length of 40, but only 25 bytes is used (by the version defined by older clients). Newer clients can recognize that the actual length of the packet is longer and know to take some action with the rest of the data. Nothing has to be done with special markers or anything like that.
If you do that, and you change the protocol a couple times, you end up with a bunch of stuff glued to the end as well as a bunch of stuff that's undefined. It turns into a mess. Look at SID_CHAT_EVENT -- half of it is blanked out. And what if different (future) clients require different things (like the way War3 requires a server signature)? You can either glue it to the end and hope something else doesn't need to be changed, or you can create a new packet. Both options are bad.
If they used XML for the communication (or JSON or whatever), they could add/remove fields for certain games fairly easily, and modify the protocol for some versions without breaking older clients.
I don't know if they'll do it or not, but I certainly don't think it's "pure stupidity", which is what I was originally responding to.
-
That's a pretty poor argument for XML, iago, because you can apply the same logic to a binary protocol: They could make the same packets do different things for different clients, but it would be just as silly as using XML.
The real advantage of XML (or JSON, or whatever) is that order is always irrelevant, which is a function of having named fields. SID_READ/WRITEUSERDATA are simple examples of how how named fields can be used in a binary protocol.
-
You can add/remove fields in a binary protocol, but you have to be very specific about it. With XML, the fields are named so you can easily ignore you don't need and parse fields you do.
-
Binary protocols can be extensible! There is one particular protocol I want to cite, but the application which it powered has been dead for years, and I can no longer find the protocol doc. The application was called HotLine, and it used an extensible binary protocol; it used application-level packets, and inside of the packets there could be any number of fields which were given type by an INT_16 (types could be "username," "flags," etc). Towards the end of its life (in the last revision of the protocol), they added a packet that allowed an administrator to view and edit *all* of the user accounts that the server maintained - to do this, they took the "read/write a single user" packet, and placed the entire packet's payload inside of a field, enumerated that over every user, and sent it as one huge packet.
The original argument against XML (made by MF) was that it generates a lot of pointless overhead - in addition to that, it takes longer to parse that overhead because it is a variable length string. I'm with you in your argument that an extensible protocol is favorable, but XML would not be my recommendation - the BNCS servers do need to take efficiency wherever they can get it.
-
I was just discussing this kind of problem with our lead interactive developer at the office. He asked whether we could use AMF3 instead of XML for our new client's new site, which is built on Flex. I'm happy to accomodate either way, but he made the point of saying: a 3kb message payload in AMF3 might correspond to a 100kb message payload in XML. You can't get that kind of bandwidth usage and speed out of XML.
-
i speculate everything is doing to use an xml-based protocol. it'll all use udp, too. udp without tcp-mimicing features. it'll use another modified SHA-1.
this is pure speculation, though.
While UDP is more convenient to process, its dangerous to use for this kind of communication since it is crucial for messages to be received reliably and in order. A better alternative to UDP is to use the new SCTP protocol. SCTP offers UDP+TCP behavior. SCTP communicates with datagrams and guarantees they are received reliably, in order and in whole. It introduces a new type of socket SOCK_SEQPACKET. Though FreeBSD's documentation indicates SCTP has support for SOCK_STREAM (FreeBSD's is the reference implementation).
http://en.wikipedia.org/wiki/SCTP
-
XML seems like overkill. Isn't it expensive to process and quite redundant? I can't remember, but I seem to remember there being security advisories in portaudit for XML parsers (very easy to mess up string processing in C).
I think SCTP + packed structs would be the best way to do it. Read an entire Battle.net packet in one call to recv(). No stupid receive logic needed to deal with fragmented packets. No parsers either.
-
oh, ok. can we still use xml though
-
oh, ok. can we still use xml though
Only if Blizzard decides to use it - but they won't.
[edit] @MF, what's AMF3?
-
oh, ok. can we still use xml though
What makes XML so appealing for this kind of application? What kind of load would a battle.net server experience processing XML messages from clients? XML is also redundant (meaning, there's extra information in the format that contributes nothing to the actual encoded data), what kind of bandwidth demands would there be? It seems like using XML would significantly increase server load and bandwidth demands. Also, current clients support a less redundant protocol that requires no parsing...what are motivations for Battle.net to support a brand new protocol instead of using the existing protocol?
-
oh, ok. can we still use xml though
Only if Blizzard decides to use it - but they won't.
[edit] @MF, what's AMF3?
AMF3 is the new Flex-based Action Message Format by Adobe. (more) (http://en.wikipedia.org/wiki/Action_Message_Format)
-
oh, ok. can we still use xml though
What makes XML so appealing for this kind of application? What kind of load would a battle.net server experience processing XML messages from clients? XML is also redundant (meaning, there's extra information in the format that contributes nothing to the actual encoded data), what kind of bandwidth demands would there be? It seems like using XML would significantly increase server load and bandwidth demands. Also, current clients support a less redundant protocol that requires no parsing...what are motivations for Battle.net to support a brand new protocol instead of using the existing protocol?
cause then i can just be like "wtb server token, or something" and it'll just give it to me? seriously. xml.
-
cause then i can just be like "wtb server token, or something" and it'll just give it to me? seriously. xml.
Seriously? Contribute something or shut up.
-
cause then i can just be like "wtb server token, or something" and it'll just give it to me? seriously. xml.
Seriously? Contribute something or shut up.
Well, you can't be too hard on me. This entire xml discussion was sparked by me not being serious. So maybe I'm good at getting others to contribute without them knowing.
But, no, I would not go with XML for future b.net modifications. There's a lot of places XML can look like a good idea and make sense but it also comes at the potential cost of having to transmit/receive a lot of unnecessary overhead. At Nortel we use something almost identical to b.net's protocol for our LTE mobile broadband (http://www2.nortel.com/go/news_detail.jsp?cat_id=-8055&oid=100250010&locale=en-US) protocol. It's more specific to individual bits where as I think b.net's seems to deal with bytes - I'm sure b.net's does this too, though. It's pretty cool that an informative packet can be like 3 bytes long. It's obvious XML would not be something we'd use here, though.
I'd think XML would be more or less for storing data and then reading that data - and using some other protocol to xmit. (if it's a lot of data)
-
cause then i can just be like "wtb server token, or something" and it'll just give it to me? seriously. xml.
Seriously? Contribute something or shut up.
Well, you can't be too hard on me. This entire xml discussion was sparked by me not being serious. So maybe I'm good at getting others to contribute without them knowing.
But, no, I would not go with XML for future b.net modifications. There's a lot of places XML can look like a good idea and make sense but it also comes at the potential cost of having to transmit/receive a lot of unnecessary overhead. At Nortel we use something almost identical to b.net's protocol for our LTE mobile broadband (http://www2.nortel.com/go/news_detail.jsp?cat_id=-8055&oid=100250010&locale=en-US) protocol. It's more specific to individual bits where as I think b.net's seems to deal with bytes - I'm sure b.net's does this too, though. It's pretty cool that an informative packet can be like 3 bytes long. It's obvious XML would not be something we'd use here, though.
I'd think XML would be more or less for storing data and then reading that data - and using some other protocol to xmit. (if it's a lot of data)
I thought the goal of a troll was to provoke anger in a discussion by carefully crafting a point that is controversial and flaw filled as to entice response. I don't think anybody is religious enough over battle.net protocols to be angered by your comments.
-
I thought the goal of a troll was to provoke anger in a discussion by carefully crafting a point that is controversial and flaw filled as to entice response. I don't think anybody is religious enough over battle.net protocols to be angered by your comments.
Look what thread you're in, tard. I wasn't trying to make anyone angry and I was hardly trolling. I was sort of poking fun at how ambiguous the topic of this thread could be. Ofcourse anything anyone can do right now is speculate and it's kind of dumb to ask for other people's speculations because it's all going to be wrong right now. This thread would end up being a collection of wasted time, which I admit can be fun... but I just thought this partcicular thread was funny.
My point wasn't flaw filled or controversial had my original post even had a point.
May I again remind you that you're posting in a b.net related protocol discussion thread and you're trying to tell me that nobody in this thread is interested in the b.net protocol. Alright, nslay... I'll just go be religous somewhere else while you can continue trying too hard right here in this very thread. Good day.
-
I thought the goal of a troll was to provoke anger in a discussion by carefully crafting a point that is controversial and flaw filled as to entice response. I don't think anybody is religious enough over battle.net protocols to be angered by your comments.
Look what thread you're in, tard. I wasn't trying to make anyone angry and I was hardly trolling. I was sort of poking fun at how ambiguous the topic of this thread could be. Ofcourse anything anyone can do right now is speculate and it's kind of dumb to ask for other people's speculations because it's all going to be wrong right now. This thread would end up being a collection of wasted time, which I admit can be fun... but I just thought this partcicular thread was funny.
My point wasn't flaw filled or controversial had my original post even had a point.
May I again remind you that you're posting in a b.net related protocol discussion thread and you're trying to tell me that nobody in this thread is interested in the b.net protocol. Alright, nslay... I'll just go be religous somewhere else while you can continue trying too hard right here in this very thread. Good day.
Cute :)
-
At Nortel we use something almost identical to b.net's protocol for our LTE mobile broadband (http://www2.nortel.com/go/news_detail.jsp?cat_id=-8055&oid=100250010&locale=en-US) protocol.
Incidentally, there's a whole lot of stuff behind the scenes of LTE that does use XML! It's all proprietary, though, so I can't even say what the things are called.
-
At Nortel we use something almost identical to b.net's protocol for our LTE mobile broadband (http://www2.nortel.com/go/news_detail.jsp?cat_id=-8055&oid=100250010&locale=en-US) protocol.
Incidentally, there's a whole lot of stuff behind the scenes of LTE that does use XML! It's all proprietary, though, so I can't even say what the things are called.
I haven't seen any XML used at all through several different stages of deployment, throughout the entire server software. I don't work directly with the client-side software, so I'm unsure of that. It's possible, though...
-
We use XML for tons of stuff in the RNC. I actually just finished generating a report for Nortel that is related to XML in a way I'm not allowed to disclose.
-
We use XML for tons of stuff in the RNC. I actually just finished generating a report for Nortel that is related to XML in a way I'm not allowed to disclose.
What does the Republican National Committee have to do with Nortel?
-
Ah, well. Who knows what it's for, then. I haven't seen it used anywhere. It's possible it's not actually used yet, or it's used on some other product entirely. It may be used in some process that we use, but isn't a part of these products. This is a large company after all. I only know of like... 5% of what goes on here. :P
-
It may be used in some process that we use, but isn't a part of these products.
Yeah, that is possible too. I think all of our products use XML for the thing I'm thinking of (think SNMP?), but I know that we're not the only company NT deals with. For some reason, though, I am thinking it's a standard for the industry.
-
Damn, what warz did was hilarious. Very nice haha.
Why wouldn't they just keep the protocol the same? They're doing just fine right now imo.
-
Why wouldn't they just keep the protocol the same? They're doing just fine right now imo.
IMO, they'll probably just keep it the same and just add more packets for the new features they are adding, for backwards compatibility with the current games...
-
If you think about it, wouldn't it be a good cost of money to refine the protocols there using now?
-
maybe, maybe not
-
If you think about it, wouldn't it be a good cost of money to refine the protocols there using now?
Why would that cost a lot? It's not like they need to enable packet length greater than 65kb. And the packet ID space isn't terribly full either.
-
Time spent on reworking a protocol would no-doubt cost them a good amount of money - I assume this is probably already being done, though.
I'd be surprised if I saw the exact same protocol being used.
-
Time spent on reworking a protocol would no-doubt cost them a good amount of money - I assume this is probably already being done, though.
I'd be surprised if I saw the exact same protocol being used.
Why would you be surprised? The current protocol is not inadequate for most of what they need; hence, they don't need to "rework" the existing protocol. They can add to it - but they don't need to enable that functionality in legacy clients.
-
Who knows? We'll have to find out!
-
I don't know if anyone has been keeping up with SC2 developments, but it's been delayed until "the first half of 2010." It was on Slashdot yesterday. I believe they said that SC2 is ready for beta, but they're just working on Battle.net 2.0.
-
Yep. The delay is more with the Battle.net 2.0 than with StarCraft 2, which is why it was delayed. On the upside, this makes the beta last longer.
-
Yep. The delay is more with the Battle.net 2.0 than with StarCraft 2, which is why it was delayed. On the upside, this makes the beta last longer.
o.O Maybe if I get my job I'll be transforming JinxBot into an internal tool....................
.....................................................................................
!!
-
Yep. The delay is more with the Battle.net 2.0 than with StarCraft 2, which is why it was delayed. On the upside, this makes the beta last longer.
o.O Maybe if I get my job I'll be transforming JinxBot into an internal tool....................
.....................................................................................
!!
Fuck that. Put all your effort into getting me a job.
Actually, I was thinking of applying for the tester job, and if you get a job there before I apply, I'd love to use you as a reference :P
-
Yep. The delay is more with the Battle.net 2.0 than with StarCraft 2, which is why it was delayed. On the upside, this makes the beta last longer.
o.O Maybe if I get my job I'll be transforming JinxBot into an internal tool....................
.....................................................................................
!!
Fuck that. Put all your effort into getting me a job.
Actually, I was thinking of applying for the tester job, and if you get a job there before I apply, I'd love to use you as a reference :P
I'd probably like referring you, from what I hear referrals get you money!
-
from what I hear referrals get you money!
We have that here. We've joked that our time would be best spent making up fake resumes to submit.
Referral: $50 gift card (previously, 2 free movie tickets)
Interview: $1000
Hire: $2500
Hire for a "hot job:" $5000
Oh, and they stack.