Cocoa Bindings and CALayer

Well, specifically I don't know if this is an issue with Core Animation layers or I'm doing something stupid. What I'm doing is programmatically binding a CALayer to a managed object with a particular key path. The problem occurs when I destroy that layer. It causes the program to crash. This is due to the a value of the object being updated (incremented skill points) and as a result it sends KVO messages to all listening objects. The CALayer never got unbound and thus a message is sent to a deleted object.

Well, I figured the simple fix is an unbind: call. At least I thought it was. The app still crashes.

I'm not sure what I'm going to do. My first guess will be to apply bindings via a NSObjectController I programatically create.

In any case, I'll hopefully finish the skill browser this weekend and start focusing work on the skill plan next week.

Graph Algorithm

Since I did spend more than a day on thinking about this one, I guess it's worth posting. Here's how it works:

You have 5 "levels", essentially the depth of the deepest tree in the game. Each of these have a value that indicates how far from the top do you go down before drawing a node. We'll call it the height for the sake of clarity. If a call is made to increment a level x (x being 1 through 5), a call is made to all levels greater than x. This call checks the value of the called level and compares it to the value of level x. If theirs is smaller than level x, it is set to the value of level x. Level 1 starts at the way left and level 5 is at the far right.

The first thing that gets passed in to the main recursive function is the root node.

To start, the recursive function checks to see if the node is a leaf, i.e. doesn't have any child nodes. If it is, it calls a drawing method that returns the drawing. It then increments the height of the level by the standard height of the drawing. Finally, the drawing gets returned.

If the the parent node (not necessarily the root node) does have child nodes, they first get sorted by their depth. An empty array is also created. Then, a call is made to the main recursive function for each of the child nodes. The drawings that are returned are added to the array. When all the children have been cycled through the height of the level first gets set to that of its first child's level height plus halfway the distance between the first child node and the last one. This will caused the parent node to be drawn right in the middle of its child nodes, height-wise. A call is made to draw the parent node. The array of child node drawings are then cycled through and lines between them and the parent node are drawn. Finally, the drawing of the parent node gets returned.

This method requires no recursion to make "adjustments" for over-lapping. It's a one-shot passthrough. It can be somewhat optimized to run a bit more efficiently but as these graphs aren't terribly big, there wouldn't be any noticeable time savings. I hope my explanation is clear and if anyone wants me to clarify any of it, just post a comment.

Core Animation!

Yes, I'm aware that this app doesn't have a real need for animation nor do intend to utilize any. Core Animation does have two great advantages for me: it's fast and it's easy (relatively speaking). It's been about a year since I even looked at CA so I spent much of my day doing some catching up on the basics.

The screenshot posted is more or less a skeleton of where I am now. I do plan on doing left to right tree growth. The size, color and content of the boxes will all definitely change but, after hours of fidgeting around with math, the box positioning looks right. I'm still trying to figure out how CATransform3DRotate works and getting the connecting lines' angles right. One big concern, though, is the size of the view. Currently, its static (and too big, for that matter). Ideally, it should be adjusted to the size of the tree to eliminate unnecessary scroll bars if the tree fits inside the view.

Still Working

I did laugh a bit a little when I saw this compiler error.

In any case, I haven't posted something to my blog in a while. I would have posted my tree drawing algorithm that I came up with but I'm sure it would bore people. But, just so people know, I am working on the skill planner with decent progress. I'm going to submit it when I have a decent tree display working.

Project Update

My latest submission is pretty big. Before you compile and run it, make sure you delete your database store as it will be rendered incompatible. Kind of reminds me that I need to start maintaining a change log.

There will come a point where the program will be a release version. Asking everyone to delete their database store (or having the program do it by itself) is simply unreasonable. I know Core Data has a migration manager, but it's pretty limited. Doing migration the old 10.4 way makes you wish you had more than one brain to pull a gun's trigger on. Believe me, I've been there. I think the path I'll end up taking is to have everything be stored in a preference file. I'll then simply parse in the data into the new store.

Skill Planner User Interface

To be honest, this is something I'm struggling with. Part of the reason I haven't submitted anything to the repository yet. I don't like EVEMon's interface. It's very, how should I say it... Windows-y? Buttons look ugly; The arrangement feels very out of place; The idea of being able to view the ship/item browser only if you create a new plan seems awkward; There's way more information on the screen then what's really practical. However, it works and in doing so I think it allows your the actual skill plan be arranged in a manner that the player really prefers.

So that's my challenge: try to come up with an interface as practical as EVEMon's and at the same time make it look clean and organized.

Skill planner and 10.5

I spent some of my time last night rewriting my code over to be more 10.5-ish. This included things like the @property and @synthesize stuff (forgetting the name) and fast enumeration. I've also noticed that Core Data behaves somewhat differently under Leopard. For example, transient attributes that weren't optional didn't require being set. Not the case anymore. Also, I may be wrong on this one but I don't recall being able to make relationships with abstract entities.

Getting back to The skill planner, I'm getting steady progress with it. However, I have taken advantage of BWToolkit, an interface builder palette with a slew of really useful elements. I've ran into this situation before of trying to include external frameworks into the resources directory and linking it. It's a very unintuitive way of doing it. So, for now, any releases will require that you put the framework into your /Library/Frameworks/ directory (notice there's no ~ for the home directory). It's something that'll definitely be fixed.

To Tiger or not to Tiger:...

...That is the question:

Whether 'tis better to put up with

The half-hearted support of ten point four,

Or to say enough is enough to my problems,

And by stopping support end them? To stop: to kill it;

No more; and by killing it to say we end

The head-ache and the future annoyances

Programming this app will cause, 'tis a desire

Greatly wanted. To stop, to kill it;

To kill it: The chance to relax, there's the reward;

For in this relaxation what efficiency may come

When we stop worrying about the limited APIs,

Makes us stop to think: there's the respect

That makes problems of backwards-compatibility;

For who would bear to play EVE painfully

The lack of dock icon, not able to command-tab to it,

The horrid graphics, the constant screen breakup,

Not to mention everyone mocking you,

For using a computer so long

With an out of date OS. What programmer would suffer,

To type so many extra lines of code,

But the clamor from stopping Tiger support,

The unknown users from whose numbers

No Mac EVE player knows, it confuses the mind

And makes me want to take on this burden

Than to limit this app to a smaller base?

Thus conscience does make the programmers struggle;

And thus the supposedly reasonable solution

Is made unclear by the desire to support everyone,

And the possibility of leaving players stranded

With this idea, most players don't seem to mind,

They'd already be supported. - How selfish!

The Mac players! Valued are you in your opinions,

Help me make the right decision.

Edit: Decision made: Not to Tiger.

While the beta build is still there for those who want to monitor your current skill in training progress I have decided that all future versions of this app will be Leopard only. EVE itself hardly runs well on Tiger (even though it technically is possible) so I see a lack of demand for it. There have been a few people suggesting I should move it up to 10.5 only. Nobody is really asking I should keep it 10.4 compatible.

Also, I'm missing out on a lot of API calls. For example, additions to NSExpression would make querying for data much faster in some cases. There are certain Interface Builder components I can now take advantage of.

However, the biggest factor lies in testing. All three of the Macs I have kicking around my house have Leopard installed. I simply don't have a way of testing if it works well in 10.4.

BETA!

After slaving away my friday afternoon and evening without a drop of Sam Adams, I think I have a pretty stable build of the application. If you go to the SourceForge link, you'll notice the zip file is there and is ready for download. No need to download the source and compile. I'd really like to hear from as many users as possible. I can only test so much on my own.

Starting next week I'm going to start work on the skill planner. If you have any ideas of making a more intuitive interface for it (I'm not a big fan of EVEMon's) I'll definitely hear you out.

Temp Icon and File Hosting

I spent some time working on a new icon. To be honest, I'm not that great of an artist so I took a photo of what I wanted it to resemble and used Illustrator to generate the paths. I know what I have now is nothing like my original idea but people do change their minds all the time, right? The blank blue page is supposed to be a blueprint of an EVE ship and the white page is supposed to have a bunch of financial charts.

I'm somewhat unhappy with SourceForge's file release system. It's just simply unintuitive and very hit-or-miss. Poking around, I did find a decent solution: Google Sites. They offer 100 MB of free storage space and I doubt I'll be causing any bandwidth issues. I'm thinking of possibly putting up my builds on this site. I know it's a huge inconvenience to ask people to download the newest XCode and figure out how to use it just to run this app. My main concern, though, is SourceForge's file distribution policies. I really can't find any details about it. I imagine, since this is a GPL license, that I can host it wherever I want but I do want to be sure.

So, in the mean time, I did set up a Google Site that I'll be using specifically for file hosting. For now, the only thing up there is my icon file. If anyone wants to look it over and modify it to make it more Mac-like with gradients and all, be my guest. I'd rather not spend my weekend working on such a minor detail.

Open Thread

Post away any thoughts and experiences. I'll be monitoring this thread closely.

Icon Postponed?

Well, I though I'd give it a shot at designing my own icon. I had Gimp installed on my iMac already so I launched it. Got prompted that I need X11. I scoured around my house for my Leopard install DVD, found it and popped it in. All of a sudden I hear a clicking sound coming from the DVD drive. I didn't think too much of it since this drive was replaced once under AppleCare and a previous noise problem didn't get fixed. During the installation process I get an error saying the install package couldn't be verified. Tried again, same problem. Dirty disk? I popped it out to clean it and lo and behold, I discovered the problem. The clicking sound was actually the sound of the DVD chipping. A chunk of the DVD was just about broken off. Really pleasant. So, no X11 means no Gimp which may mean no Icon.

Sigh... Here I go again with AppleCare.

Update:

Apparently my warrantee expired 6 days ago. This kind of thing seems to happen to everyone, huh? I got passed around to a couple of representatives (I didn't have to ask for any supervisor) before a third was able to make an exception for me. That and they're wiling to replace my Leopard DVD. I've always had great luck with AppleCare and every experience with them has been pleasant. Hands down best customer service.

Memory Leak

... is found! At least the one related to the constant updating of the current skill in training.

I literally spent my whole afternoon tracking it with the help of Instruments. Very frustrating, to say the least. The culprit? The NSUndoManager with its unlimited levels of undoing. I was pretty much asking for the memory to be eaten. Even Objective-C's own garbage collector wouldn't have caught this.

For now I just set the context's undo manager to nil. Nothing I'm doing requires any kind of undoing. If later on this needs to be changed, it's a relatively simple fix.

Tomorrow I'll try to focus on finding any memory leaks related to updating the core data store. Hopefully this one is easier to track. And with the undo manager out of the way, I won't run into the same problem. On the plus side, I'm hoping, not having an undo manager will speed up performance by a noticeable amount.

On a side note, I'm finding that at this stage of development I'm spending far more time debugging and optimizing then I am adding new features. I hope that after a beta release this will change for a bit as I find adding new stuff is far more fun to do.

Commenting

I know people hate the tedious process of registering to post comments so for now I've enabled anonymous commenting. If I get hammered with spammers or things get out of hand, I'll re-enable the registration requirement.

Icon and Update

As I'm doing finishing touches for a beta release, I'm hoping someone can design an icon for me. Specifically, I'm looking for a picture of an EVE ship flying towards you and shooting (I'm thinking Caracal as the best looking one) and the letters 'EMS' above it. Lastly, it has to have the shiny Mac icon feel to it. If anyone can think of a more appropriate one, I'd be willing to consider.

Oh, and the current build as of this posting is very stable. The only downside is that every time the application launches, it forces an update on all of its XML files. For the skill in training and skill queue files, this isn't a big concern. However, for the character sheet, it does tack on several seconds per character in loading time. I'm somewhat torn with what to do.

I also ran this program overnight. For every update, I'm loosing about half a megabyte of memory to leaks. Personally, I find that acceptable for now. Using Instruments to track the leaks has been tediously difficult as it takes over 5 minutes to launch the app and more often then not, Instruments crashes.

Core Data and Transient Properties

For a while, I've been having this problem show up in the debugger: 

[ valueForUndefinedKey:]: the entity EVECDCharacter is not key value coding-compliant for the key (null).

I suspected I knew where it was but could not, for the life of me, figure out why it was crashing. My mistake was that I wasn't paying attention to the fact that the line I (correctly) thought was causing the crash was done in a separate thread. So when I did realize this and took a look at the stack trace, the above picture is what I got.

I don't know what I should be more concerned with: The huge stack size or the fact that a transient value was being requested. Line #5 is specifically where the call gets made. In that call, a request for the skill is requested. However, the skill doesn't get set until later in the skill queue API parsing.

For now the easy solution would be just to do a simple check to make sure the value exists. It is something I'd like to find an explanation for, though.

First Beta Progress

The training queue is just about finished, hopefully. When this is done, I won't be working any new features. I'll be finishing up existing stuff and try to do as many tests as possible for different scenarios.

One big internal change I'm doing is with the timers. I've completely eliminated the one for the API downloader. I've also eliminated all the "secondary" character ones, i.e. the skill in training and skill queue. These will automatically update on a character sheet update.

The last big change, and the one I'm worried about the most, is the forcing of updates whenever the application launches. The downloading and parsing of XML files does take some time and it'll take a considerable amount of time if the person has a large amount of characters. I am pleased to report, however, that when the app detects a skill is finished being trained, it updates the data without errors.

Training Queue

I looked at the training queue EVE has and decided that duplicating it would be the best way to go. I spent just about the whole day yesterday working on it and am surprised how fast it was. Building a solid foundation with file handling and the Core Data model is starting to pay off.

What's new to me this time around is making a custom view. I've never had to make one before but found that working with NSBezierPath is actually very easy. Now I'm at the point where I just have to draw the scale.

As I'm adding more and more stuff, I am somewhat concerned about how much more time it's adding to the launch time of the application. When I first started this this app, I was amazed how fast everything loaded. Almost instantaneous. Now I've seen it go as high as 10 seconds. Still not a lot but that's only with 2 characters and with several more features to go.

Stability Improvements

Stayed up until 6 am doing major changes to the app. The character tab's outline view is now completely populated by bindings to an NSTreeController. What really made this work is transient properties in Core Data.

Looking at the picture above, I introduced two new entities; an abstract EVECDDisplayElement entity and an EVECDLearnedGroup entity. While instances of the latter are created and saved into the store, none of its fields are. I didn't want to overpopulate the store with redundant data. What makes this works o beautifully is how easy it is to bind everything via the learnedGroupsOrSkills key path. The actual code to generate these new learned group entities is less than 20 lines and execution time of it is near instantaneous. 

From an MVC perspective, I see it somewhat of a compromise. Part of me tells me view related information shouldn't belong in the skeleton of the model. The real "model" is the store. The "skeleton" is the xcdatamodel file. Kind of hard to try to keep the terminology straight. On the plus side, however, I'm going to do a lot less worrying about KVC notifications and repopulating the table.

The other major change is how I'm locking the managed object context. Before, I figured using @synchronized did the job for me. In reality, I misunderstood its use and now use context's own lock and unlock methods.

Hopefully a beta release will be ready this weekend.

Edit: Err.. yeah, what picture?

Multiple Characters Issues

At the moment, problems revolving around having multiple characters seems to be my problem. I keep getting "multiple validation errors" whenever I try to launch the app with two characters. Not very intuitive when you're trying to find what exactly those errors are.

I was having quite a few random crashes over my dataSource calls for the character tabs' learned skills outline view. After discovering it was an issue that had to do with calling reloadItem: while at the same time a delegate call to get the row height is called, I was throughly frustrated with these dataSource calls altogether.

I took a few hours to convert the Core Data model over to support a bind-able hierarchy. I then set up a tree controller in the character tab's nib and bound it to the newly created entities. These entities actually happen to be transient entities. I didn't want to store redundant information.

Lastly, I decided to take a gamble and have every form of updating be done on the main managed object context. I'm not seeing any multithreading errors but then again, I think I have a pretty decent context lock system in place.

One of Those Days

Rather than pushing towards getting new features out, I decided to take some time to go back and fix some bugs.

I do my work on two computers; an iMac and an iBook. Most of my time I'm doing my work and testing on my iBook. Most of my testing recently has been done with one character loaded. Earlier today I decided to do it on my iMac instead, for which I have two characters saved in my preference file. When I launched the app on it, I realized how much more unstable it ran. NSNotifications were being called twice (no idea why) and a character's skill in training was being showed on another character's tab.

That's the start of my problems today. I'm pretty sure I'll run into more.

Project Update

I've made quite a few changes with my last submission I did just a few minutes ago as of this writing. I'll try my best to list them.

Noticeable Changes:

• Smaller application size. Things were way too big and spaced out so I compacted much of the information.
• Learned skills outline view now has learned skills shifted farther to the left. Ended up doing something Apple probably doesn't recommend but oh well.
• Current skill in training information now showing. Untested in circumstances such as what happens when a skill finishes training.
• Dynamically estimates the number of skill points your character currently has.

Under the hood changes:

• Reduction in the number of active NSTimers on the run loop. Got everything related to the current skill in training reduced down to only using one timer per update.
• API controllers now post notifications whenever they finish updating the Core Data store rather than going through a series of delegate callbacks. 
• Tighter linkage between the character data and tab controller for passing data between the two.
• Server status bindings properly display values without manually binding every time an update occurs.
• A few memory leak fixes.

Hopefully this week I can announce a beta. For now, it's almost 5 am and I still haven't slept. Time to crash.

The Skill Point Timer Problem

Note: This post is more or less a discovery of common sense solutions. When I realized the best, and conveniently the least complicated, solution it was only then I saw how far off the deep end my mind went.

The frequency of skill points updating can easily be calculated and the value can accurately be guessed. A running counter of the current skill points for an app of this nature would almost be a necessity.

Here's the simple reason why there is a problem: NSTimers just aren't reliable. From Apple's own class reference of NSTimers: "If the firing time is delayed so far that it passes one or more of the scheduled firing times, the timer is fired only once for that time period; the timer is then rescheduled, after firing, for the next scheduled firing time in the future."

So say the counter is running on the main thread and is updating the current NSManagedObject being learned. If you press a menu item and hold it long enough, it could easily cause your skill points counter to be thrown off.

One might then suggest why not do the updating on a separate thread? That would certainly solve the issue of not having user or other events interfere. But now you're creating a whole slew of new problems. First off, I have to have a perpetual secondary thread for the shear purpose of keeping a counter. This means doing quite a bit of learning on maintaining an active run loop. Then there's the problems of sending messages to the new thread. Ok, so even if I somehow manage to solve all those problems, there's still remains another unsolved one: Context locking. Currently, I have two contexts; one for displaying data and one for updating. That was tricky enough to figure out. If, for whatever reason, either of them get locked long enough, I'm back to the same problem. Adding a third one just for the sake of updating one object on a timer is just plain silly.

Then I thought of a 3rd idea. Why not a 3rd thread! The main one for user interaction; A second one for updating values; Then the third to have a timer that only updates a variable. The second one would query the third one for the proper value. If, for whatever reason, the second thread pauses too long, the counter will still continue. Brilliant!

And then it dawned on me. Why am I using an NSTimer for keeping the accuracy of the skill point values? And why does something so simple have to have multiple threads? The real solution is just create a static date to reference for computing the amount of skill points. Have a timer be set up on the main thread to call an update method. This method takes the date, computes the skill points the character would have based on that date and updates the object. If somehow the timer doesn't fire one or more times consecutively, the next time it does it will update to the appropriate values.

Cocoa Bindings and NSWindowController

Having a pretty eventful weekend so I haven't had much of a chance to update.

While figuring out the kinks to removing characters, of which I think I've got all figured out, I ran into an interesting problem. My character tabs are handled with an NSWindowController tied to a nib. This nib's file owner object (CharacterTabConroller object) contains outlets, one of which is tied to an NSView object (rather than an actual NSWindow). In this nib there is also an NSArrayController.

Here's where it gets interesting. Apparently, in some way, this NSArrayController adds a retain count to the NSManagedObjects (in my case it was the character's learned skills) it's controlling. Originally, I set it up so that the character's data gets deleted first. However, after getting a series of bad access crashes, I noticed in the debugger there was a call to NSAutounbinder. Ok, so maybe I should be releasing the NSWindowConroller first and then removing the actual character data? Nope. Still crashing.

After a bit of digging on the internet, apparently you actually have to unbind the NSArrayController before releasing its enclosing NSWindowController. Personally, I think this is counter intuitive.

On a Personal Note

Completely unrelated to my development but I figured this would be a good place to start looking.

Currently, I don't play much because of developing this app. Hopefully that'll change after a month. In the mean time, though, I am looking for a decent sized, established corp. A corp that actively helps each other with missions. Preferably a corp that has stations in 0.0 sec.

Where I am now there's only about 5 or so members. While I think they're a great bunch, chances are when I log on they probably won't be on. Part of what makes an MMO fun is the people.

So yeah, I'm looking to move on up. I only have a few month's worth of skill points accrued and not too much in assets. If I was to join another corp, though, I wouldn't be looking for handouts. So, if your corp fits my description of what I'm looking for and its willing to take me on, post a reply on how I could get in touch with you.

Tracking Skill Points and Removing Characters

As per title, these two things are part of my work in progress. Normally, I'd hold off on getting new features in but my weekly write up on my progress does look better when it appears that I've been tackling more problems.

For skills currently training, I first calculated how many seconds it takes to get 1 skill point based on the character attributes and skill being trained and then set up a timer to fire at that interval to update the value. The learned skill managed object's skill points learned value gets updated also at that frequency and I set the context to be updated once every 20 seconds. I'm not sure I'll stick with that value but I think it's a pretty reasonable number. Also, while, in the process of typing this, I don't recall writing in code to stop the timers once the accrued skill points hit the requirement for the next level. Hmm...

Removing characters is a work in progress now. For a while what bugged me was the retain count on some of these objects, some being as high up as 5. I went in and wherever I could, I put in releases. I then realized my setDelegate: methods were also affecting the retain count so I fixed those. Finally, after a while of probing, I came to find out NSTimers were also adding to the retain count, forcing make methods to kill the timers before sending a release message to the object. I'm now at the point where I'm passing a release message to an NSXMLDocument but its causing me to crash every time I access that object. It's late, though, so it'll be tomorrow's project.

As always, delete the contents of ~/Library/Application Support/EVE Mac Suite/ before running this app. I may later one, for my own sake, too, create a way to allow the user to completely wipe the cache and force the app to rebuild its database.

On a final note, Sam Adam's Cherry Wheat is damn fine beer.

NSNotifications

I've known about NSNotifications and what they can do but I apparently was completely oblivious to them when writing a lot of my code. In writing a response to Erik on my previous post I realized that I've been making my life harder with some of my programming. For example, before you can view what a character has learned, the character sheet xml needs to be processed. Before that can happen, though, the skill tree xml file needs to be processed. The way my code works is via a series of call stacks. While debugging I've noticed the call stacks get as high as 6 stacks and maybe even more due to a series of delegate callbacks.

While the application may still run fine with that, there is one big problem: it's blocking the run loop.

With a large call stack, the application could be creating a large amount of autorealeased objects that gobble up memory and won't get released until the application's main thread hits the run loop. I don't foresee such objects being around that long but what if you're in the middle of intense EVE PvP and you suddenly get a deadly lag spike because this app was running in the background and suddenly required a lot of memory? You wouldn't have guess it was this app but that wouldn't change the fact that this app made your EVE experience worse.

The more noticeable problem, though, is that huge call stacks that last several seconds prevent events from being received on the run loop. The application essentially hangs until it finishes its task.

Sometime this weekend, hopefully, I'll be making a pass through some of the code and changing it around to use NSNotifications. I might also post up some diagrams about the logic flow of my code. Its getting complicated to the point where I'm occasionally struggling to keep it all straight in my head.

Oh, and as for the NSManagedObjectContextDidSaveNotification thing you mentioned, Erik, I did some research on it. NSNotifications on a spawned thread never hit the main thread. NSDistributedNotifications are needed for that and I'm not sure I want to go there. A general did save notification might be too broad of a notification, too. For example, updating a character's learned skills table view is done via a notification. If only character A's data changed, there isn't any need to update character B's table view. What I might end up doing is making my set of notifications based on what kind of data gets updated.