Archive for January, 2008

Lobby server nearly done

Wednesday, January 30th, 2008

Man what a crunch.

Here’s the capabilities of what it can do on the PC:

(~) Quit
(A) Login
(B) Set TitleId
(C) Logoff
(D) Register an account
(E) Retrieve password recovery question from handle
(F) Retrieve password from password recovery question
(G) Update account
(H) Change handle
(I) Download friends
(J) Send add friend invite
(K) Accept add friend invite
(L) Decline add friend invite
(M) Get friends list (print them)
(O) Download ignore list
(P) Add to or update ignore list
(Q) Remove from ignore list
(R) Get ignore list (print it)
(S) Download emails
(T) Send email
(U) Delete email
(V) Update email status
(W) Read emails (print it)
(X) Send IM
(Y) Create room
(Z) Download rooms
(0) Get All rooms (print them)
(1) Leave room
(2) Join room
(3) Room chat
(4) Room invite
(5) Set ready to play status
(6) Kick room member
(7) Set room invite only
(8) Set room allow spectators
(9) Change room slots
(!) Grant moderator
(@) Start game
(#) Quick match
($) Cancel Quick match

It’s all about 15,000 lines of code (probably a quarter of that copy/paste hehe) written mostly over 6 or 7 days. It’s late so I’m making a lot of mistakes.

Tomorrow I will document the headers, test it, and move some files around. Then I can release it as the first beta.

3000 lines of code in 2 days

Monday, January 28th, 2008

This lobby server is turning out to be much more involved than I originally thought. I have to write every query between 6 to 11 times.

For example, to add a friend to the friends list for a user I have to:

1. Query the user for the friend
2. Validate the inputs on the client (we are connected, etc)
3. Serialize the inputs and send them to the server
4. Deserialize the inputs on the server
5. Check the inputs for sanity (don’t add yourself as a friend, you are connected, …). If bad, reply back.
6. Serialize the inputs again in database format, and send to the DB thread.
7. Deserialize the output from the db thread
8. Parse the output from the db thread (doing the actual work of maintaining the internal friends list).
9. Serialize the reply back to the client.
10. Deserialize the reply from the server on the client.
11. Parse the reply (modifying our friends list), and call the output callback for the user.

That’s not even counting that this relies on another previous query to ask/authorize a friend invitation.

This would be even harder and more involved if I had not yet already wrote and tested the DB queries in 3 other projects!

So I did 9 client commands in 2 days for about 3000 lines of code. I have like 50-60 more commands :(

The good part is the only painful commands left are the ignore list (which is easier than friends) and sending emails. Friends was probably the hardest to do.

I’ve sort of skipped per-user permissions for now, and moderator commands. Those will also be very involved and painful.

Reflection failure :(

Friday, January 25th, 2008

I feel pretty bad. For two weeks I have been trying to get reflective floors working in Gamebryo. At first I tried just rendering from a different camera angle. It never really worked. So I next looked into portals. But after a couple of days where I actually figured them out, I emailed support to clarify some questions and they said that portals were just for culling, and were not true portals. So then I found a sample where they have reflection. But the code was split among 3 files. So I extracted the part I wanted out but it didn’t work. It took me a couple of days to get a pink texture floor. Then a week to get a texture that moved with the camera, but was distorted and totally wrong. Then I updated the code after not looking at reflection for a couple of days and I couldn’t even get the texture to update anymore.

So I threw in the towel and told the boss to give it to someone else with more graphics experience.

* EDIT * Gamebryo support was helpful through this process. This shouldn’t be taken as a knock against Gamebryo, just my own lack of graphics knowledge. * END EDIT *

While the world doesn’t need another graphics engine, these experiences motivate me to continue work on Rak3D. There’s two features I want to have that I haven’t seen before:

1. Rather than letting users program most features themselves via low-level constructs, Rak3D would include every feature I can imagine, in modular classes. So reflection would be one line of code, the shader would be a multi-part super-shader that does normal, parallax, light mapping, etc. This would be organized into a two-level system, low and high. The low level system is typical of what you see in 3D engines right now. The high level system is like RakNet’s plugins, where they expose direct features. What I like about this is stuff just works, there is no debugging needed.

2. A big problem I have is stuff just doesn’t show up and I have absolutely no idea why (as above). I want to include an interactive on-screen debugger where I can look at my scene heirachy as a tree, which will indicate with colors what is visible. And I can select any object and get full details on it, esp. why it is not visible on screen, or why it is using a default or missing texture.

RakNet features video

Tuesday, January 22nd, 2008

I think the RakNet website is too programmer-centric. If you look at other sites like FMOD they don’t have a single word about their technical superiority, but instead list their features. To remedy this I’ve made a RakNet Major Features Video. It took me about 4 hours to record, since I restarted many times. The first successful recording was 40 minutes long, I cut it by half twice. The current video blows through the features very fast, but I think is more exciting for it.

Diebold machines flipped Obama and Hillary votes in NH

Sunday, January 13th, 2008

“Thus we can say with a high degree of certainty that there is a relationship between the counting method and the election results. The probability of the aforementioned discrepancies occurring by chance is less than 1 in 100 billion.”

So the chances of rigging are 99.9999999%. Obama won, Hillary lost.

How hard can it be to write a freaking voting machine? And if you want accurate results…


What do you expect when you have private corporations with vested interests write closed systems?

4 years ago they were talking about this exact same thing, vote flipping. I won’t bother finding the links but there are all over the place. I don’t mean half-assed speculation either, there is good statistical evidence for this and proof it can be done.

The only good thing now is they found it before we got a new president, rather than months after he is already in office.

Ranking server done

Saturday, January 12th, 2008

I’ve finished writing a ranking server for RakNet. It is a C++ interface to a PostgreSQL database that stores a table that contains a list of matches between two participants. It’s very general – participants, games, scores, ratings, are just numbers. You can store a text string and arbitrary binary data for each match. It uses my functor system so calls are asynchronous.

There are currently 4 functors:
SubmitMatch – Submits the match results between a pair of participants.
GetRatingForParticipant – Get the most recent rating for a single participants.
GetRatingForParticipants – Get the most recent rating for a list of participants.
GetHistoryForParticipant – Get all the matches and the data therein for a particular participant.

I store IDs as a pair of unsigned integers. The intent is that the primary integer can represent a foreign key. The secondary integer can represent a sub-type. In Galactic Melee for example I store rankings for both players and squads, so the secondary integer would differentiate between these.

The class is designed in such a way as to be easily extensible while still easy to use. Generally speaking the more extensible something is, the harder it is to use because there are more pieces involved, so you have a lot more to learn. This is true of Gamebryo for example, which has good division of code but then requires a lot of steps to do anything. I wanted to avoid this, so went to a lot of extra work to make as much automatic as possible.

One nice thing is you can add more queries without changing the original source code. This is because each query is just a derived class.

The current table format is:

CREATE TABLE rankingServer (
rowId serial UNIQUE,
participantADbId_primaryKey integer NOT NULL,
participantADbId_secondaryKey integer NOT NULL,
participantBDbId_primaryKey integer NOT NULL,
participantBDbId_secondaryKey integer NOT NULL,
participantAScore real,
participantBScore integer,
participantAOldRating real,
participantANewRating real NOT NULL,
participantBOldRating real,
participantBNewRating real NOT NULL,
gameDbId_primaryKey integer NOT NULL,
gameDbId_secondaryKey integer NOT NULL,
matchTime bigint NOT NULL DEFAULT EXTRACT(EPOCH FROM current_timestamp),
matchNotes text,
matchBinaryData bytea);

Here’s an example of calling a Functor

GetRatingForParticipants_PostgreSQLImpl *functor = GetRatingForParticipants_PostgreSQLImpl::Alloc();

printf("Get the most recent rating for all participants\n");

printf("Enter the primary ID of the game (int): ");
functor->gameDbId.primaryKey = atoi(inputStr);

printf("Enter the secondary ID of the game (int): ");
functor->gameDbId.secondaryKey = atoi(inputStr);

/// Puts this functor on the processing queue. It will process sometime later in a thread.

This is all part of my plan to write a complete lobby server, of which ranking and match tracking is one component.

Function threading for database calls

Thursday, January 10th, 2008

I’m working on adding a ranking server and a lobby server to RakNet. One of the problems this entails is that database calls are blocking. It’s possible that the ranking server or lobby server will be doing other things at the same time (such as running the game), especially for smaller games. This means that database calls have to be run in a thread.

I spent a few hours setting up the ranking server calls to execute in a thread by copying the input parameters, doing the function, copying the output, and calling the result back in the main thread. But this was really a pain in the butt. It’s not the first time I’ve had to have done it. And I’ll have to do it again, but much more so, for the lobby server.

This morning when I woke up I thought of a solution that so far is turning out well. Create one thread that takes a stream of functors, does processing on those functors, and when done calls a function in the main thread via a callback. The functor itself will have as member variables what was formerly the input and output parameter lists. This avoids the necessity of copying parameters. And since it is up to the user to derive the functor, they can add contextual user data or extend it as they wish. (As opposed to passing a void* and worrying about allocation).

Here it is, leaving out the implementation

/// FunctionThread takes a stream of classes that implement a processing function,
/// processes them in a thread, and calls a callback with the result.
/// It's a useful way to call blocking functions that you do not want to block,
/// such as file writes and database operations.
class FunctionThread

/// Starts the thread up.
void StartThread(void);

/// Stop processing. Will also call FunctorResultHandler callbacks
/// with /a wasCancelled set to true.
/// \param[in] blockOnCurrentProcessing Wait for the current
/// processing to finish?
void StopThread(bool blockOnCurrentProcessing);

/// Add a functor to the incoming stream of functors
void Push(Functor *functor, void *context);

/// Call FunctorResultHandler callbacks
/// Normally you would call this once per update cycle, although you
/// do not have to.
void CallResultHandlers(void);

/// A functor is a single unit of processing to send to the Function thread.
/// Derive from it, add your data, and implement the processing function.
class Functor
virtual void Process(void)=0;
/// Called from FunctionThread::CallResultHandlers with wasCancelled
/// false OR Called from FunctionThread::StopThread or
/// FunctionThread::~FunctionThread with wasCancelled true
virtual void HandleResult(bool wasCancelled, void *context) {};

Here’s one of the smaller functors. It gets a list of players given a game database id.

/// Given a particular game, get ratings for all players
class GetRatingForPlayers_Functor : public Functor
/// Identifies in the game in the database with a primary
/// and secondary integer key. User defined.
RankingServerDBInterface::EntityDBID gameDbId;

/// List of players and rankings for a given gameId
DataStructures::List < rankingserverdbinterface::RatedEntity > outputList;

Before I had to

1. Add member functions for every operation
2. Add a callback for every member function
3. Copy the input parameters for every operation and store them in an input list
4. Copy the result of the operation and store them in an output list.
5. Create a thread to do the functionality
6. Create interfaces for every operation, that actually does the work in the database

Now I can
1. Create a functor for every operation
2. Implement the functor
3. Pass the functor to the FunctionThread instance

Another nice thing about this is I can reuse the same FunctionThread for both the ranking server and the lobby server. The old way would have created a thread for each of those classes.

Uploading RakNet to Sourceforge

Wednesday, January 9th, 2008

It’s about time I put RakNet under source control. I wasn’t able to figure out how to host my own SVN server so after a few tries I got it approved by Sourceforge.

The upload process is very complicated and confusing. Right now I am stuck here:

I created the following directory structure.


Then I unzipped, which has all the distribution files, to RakNetSVN/trunk

Then I right clicked on RakNetSVN, selected import for TortoiseSVN, and entered for the subversion server. I get a similar error to the one above.

Anyone know what this means?

Am I supposed to run Putty?

** EDIT **

‘renchap’ helped me with this.

The correct URL is
You have to first checkout to the directory with your files.
Then you add your branches, tags, and trunk

He also helped me setup a bunch of other cool goodies.

1. Fixed the makefiles
2. Release notes automatically emailed, sent to RSS, and sent to IRC
3. Stopped hosting the Wiki myself

Sourceforge is very powerful!

Got Cox Business Cable

Wednesday, January 9th, 2008

I needed a static IP so installed Cox Business Cable. It’s $200 to lease the modem, $300 to install, and $120 a month for 1 MB upload and 8 MB download. I thought that was a pricey, so I asked the sales rep. if there was anything special about the modem. She said there was, but actually it is just this . Prices for this modem on Google range from $50 to $78. The installation was two guys coming to my house, unplugging my existing modem, and plugging the new one in.

I think we need some competition in the Irvine area.

Fair tax

Tuesday, January 8th, 2008

Economics is a fascinating subject to me. I heard about Fair Tax on the radio today, a tax scheme supported by Huckabee. It is a national sales tax with a fixed amount returned to taxpayers every year to offset it from being a regressive tax. It is supposed to be a simpler replacement for income tax.

I found an informative and honest article on factcheck about it. The main points I got were:

1. The proposed rate of 23% is deceptive because it is 23% of the total cost of a good/service including the tax, not 23% added on as is customary for sales tax. This makes the number look smaller than it actually is.

Let f = the fairtax percentage, or the tax applied to the total cost paid. If we paid 50 cents on a $1 item, then this would be 50 cents out of the total price $1.50, or .3 (one third)
Let p = The pre-tax, e.g. percentage of the tax actually paid. If we pay 50 cents on a $1 item, then this would be .5 (one half)

The fairtax is then the percentage of the tax actually paid, divided by the total amount paid. Here 1 represents the percentage of the retail price, while p is proportionate to that.

f = p / (1 + p)
f = 1 / (1/p + 1)
(1/p + 1) * f = 1
1/p + 1 = 1 / f
1 / p = 1/f -1
1 / p = 1 – f / f
p = f / (1-f)

So for a fairtax rate of .23, the equation yields .2987. With significant digits, this is .30 or a 30% tax.

2. The 30% rate is a low-ball figure and likely insufficient to break even. 35% is more realistic. It also assumes 100% compliance, which is unrealistic.

3. This tax applies to all goods and services (food/rent) not just things we are normally taxed on.

4. Criminals tend to end up paying tax when they spend their ill-gotten gains, where before they did not.

Some points the article didn’t bring up, but occur to me, are:

5. Business as a rule try to avoid/hide making a profit since they have to pay tax on profit. In the game industry this is evident in the publisher business model. The typical model is to give very bad deals to game developers such that publishers make nearly all the profit on a blockbuster. However, this is offset because 90% of games fail. The failed games are write-offs and for tax purposes make a queer sense because then taxes do not have to be paid on the profit from that one blockbuster. In the national sales tax model publishers would be taxed on expenses, rather than income. The first thing I believe this will do is to make it now make sense to buy and run moderately profitable companies. This would otherwise be undesirable because the money would be tied up in slow gains, rather than shooting for the big payoff. It’s sort of like playing the penny slots vs. the $20 slots. The second thing I believe this will do is to make publishers more risk-adverse, since they no longer get ~30% of their money back.

6. Currently one of the difficulties of lying on taxes are that there are two records. I don’t want to report my income because my incentive is to not want to pay taxes on it. My employer wants to report my income because their incentive is to use it as a write-off. So both of us have an incentive to rat each other out, and thus both of us stay honest. But when taxing expenses only, the record is one-way only, with a strong incentive to lie about it.

7. According to this site it costs taxpayers 22 cents for every dollar collected. This system is much easier on the taxpayers, since you no longer have to save receipts and so forth. It’s just a higher payment on your final bill.

8. The goverment is very good at hiding taxes. Rather than hitting a 100K earner with a 33K tax all at once, it’s taken out evenly over bi-weekly paychecks over the year. And the cost is split up into different categories (Social Security tax, etc), so it’s hidden even further. What is funny is people are excited to get a refund of their own money back, while neglecting to notice that they are just paying less. My point is that most people don’t notice just how much they are getting taxed on income tax alone (don’t forget you also pay inflation tax, tarrifs, subsidies, etc!). But with Fair Tax, that $40,000 car you wanted is now listed for $52,000 and you are going to be freaking outraged at the $12,000 tax on the car. This is a good thing, since more transparency is good, and I think will reduce compliance and put tremendous pressure on the goverment to reduce taxes. Or then again maybe not so good, since they will just hide the taxes elsewhere.

I think I would support this system. It’s more transparent, it’s simpler, and the economic distribution is roughly the same. Of course I still hope Ron Paul wins, since no income tax is even better.