Introduction
If you are reading this, chances are that you’ve got some experience with personal finance software. You’ve probably tried a lot of apps until you found MoneyWiz… and you probably noticed that MoneyWiz is the only finance app that offers automatic live sync.
But have you ever wondered why? As a matter of fact have you ever wondered why so few apps in the world actually offer a decent sync in year 2012? If you have, read on… I will try to explain how these things work and how our own syncing service, Sync Everything, works.
How apps work with data
Every data-heavy app, such as a personal finance app or a to-do app, works pretty much the same way when it comes to storing and reading its data.
Such apps store data into database files. A database is basically a file, that allows us – the developers – to organize the data into tables with columns. Pretty much like an Excel spreadsheet but much more complex.
There is one more important thing here – where is this database file stored? The simple answer is that it could be stored either on your device (be it a mobile device or a desktop computer), or it could be stored online. Online storage and exchange of data is as old as the Internet, but these days we use this geeky-cool name – “cloud”. Did you notice that for the past year or so, everything has been going to the cloud…well if you ever wondered where’s that popular cloud that everybody knows of, and you don’t – wonder no more. It’s just the good old Internet.
What about syncing?
So, what’s sync? It stands for synchronization and it’s basically the exchange of data between two or more devices, until the data becomes the same on all of them.
There are two ways to sync:
You can sync directly between the devices…Bluetooth and WiFi sync do that, when you connect the devices on the same network, or just pair them with each other…a.k.a. peer-to-peer.
You can sync via an arbitrary third device. That device is called a server, and when it’s on the Internet, we call it “cloud server”. One of the devices would upload data to the cloud server, and the other will download it from it, but the two devices would never communicate directly to each other.
Syncing devices directly, via WiFi or Bluetooth is too much hassle. It’s just and old and obsolete way of doing this. Add a third device, and you will be looking into a serious waste of time. Also, the devices need to be physically close to each other for this to work. So, this sort of syncing solution is off the table…at least for us.
Cloud syncing, in the other hand, looks like the cooler thing to have. It gives more independence, it works with unlimited amount of devices, and it can be automatic. The devil is in the details though…
How do the popular sync services work?
Lots of people ask us if we’ll do iCloud or Dropbox sync for MoneyWiz…so let me briefly introduce you to how these services work and why we won’t integrate with them…
Dropbox
Lots of people know and use Dropbox, and it’s a really great service for automatic files syncing. What Dropbox does is that it gives you a place (a folder on your disk) to put your files into, and when you put them there, they are automatically uploaded to Dropbox’s servers. This of course is possible after downloading Dropbox software that will take care of the uploading and all. That same software, which is also known as a “client software” will also check periodically to see if something new has been uploaded to your Dropbox folder, but from another device, and if that happens it will automatically download it on your current device. All of this is possible because ultimately, all of your data is located on a cloud server, and all of the devices you use just collaborate with this cloud server.
iCloud
iCloud is actually a set of many services accessible with single username and password. Some of these services include IMAP to sync your emails, service to sync your address book and calendars, iTunes Match to sync your music and so on… One of the iCloud services allows us to sync the databases of apps. This is what we, the developers, can do with iCloud – sync our app database across all of the devices that a certain iCloud use has. The problem with that is just one – this is pretty much the same as Dropbox, because at the end of the day, we will be syncing the whole database file.
Why not?
You may be asking yourself now – “Why not sync the whole database file?”… I will give you 5 good reasons!
- Imagine that you have an iPhone and an iPad. You may want to enter some transactions on your iPad and some other on your iPhone, but unfortunately one of the devices doesn’t have Internet connection at the moment. You would obviously want to have both devices sync as soon as both of them regain Internet connection. Okay, but you made changes on both devices, therefore you’ve changed the database on both devices. If we were to sync the entire database file, this would mean that either your iPad will send its database file to your iPhone, or your iPhone will send its database file to your iPad. Either way, you lose the changes made on one of the devices, because that device just received a database file that overwrites its own database file. That would be pretty annoying, wouldn’t it?
- If we were to sync the entire database file, this would mean that every time you make a new transaction, we have to upload and download the entire database file that also contains all of your transactions. If you have 1000 transactions, we are looking at at least 2MB database file size. Every time you make a new transaction your Device X will upload 2MB to the cloud and Device Y (your other device), will download them afterwards. If you make 5 transactions per day, using just 2 devices, this would generate approximately 600MB data traffic per month. That’s expensive if you are on 3G!
- Many families use MoneyWiz for their home finances. Such families often have 3 or more devices in total, that are used by different family members. If we used iCloud it would mean that all family members must use the same iCloud account on their devices, otherwise the sync won’t work. iCloud credentials can’t be set per app!
- Some people would propose to workaround the above problems by syncing manually as opposed to automatically…in other words – to ask you, the user, to tap some buttons when you want to sync… That would obviously prevent these issues, but the last time I checked we were living in year 2012… Technologies are powerful enough to allow us to do the syncing automatically for you. Anything less is just a poor excuse.
- iCloud is controlled by Apple. That means that if we went for iCloud, we wouldn’t be able to make MoneyWiz for Web and for Windows for example. That’s a limitation we don’t want.
From Dream to Reality – Sync Everything!
When we started developing MoneyWiz two years ago, one of the major things we wanted to do was proper syncing.
We imagined the perfect sync to take a minute to set up and no time at all to manage. It had to do everything automatically!
We first wanted to find an existing solution that would save us some development time, but it turned out nobody had done this before. So it took us about 2 months just to figure out how to do it and another 6 months to actually develop and test it. Some of the challenges we had were:
- It had to be secure
- It had to be scalable (i.e. allow us to have at least 100,000 active users, and then allow us to scale more on demand)
- It had to be fast
- It had to be reliable…meaning – no downtime.
- It had to properly figure out what to do in all sorts of situations, like when you have many devices, some online, other offline, and you make changes on all of them at the same time.
We knew from the very beginning that syncing the entire database won’t work well. It would’ve been the easy solution, but not the right one. So we came up with something very different… We’ve figured that instead of syncing the entire database file, we will sync the changes only. So basically we’ve decided to sync commands that describe what you do. This way, we could send these commands to all other devices running MoneyWiz, and MoneyWiz would execute them, just as if you did these changes yourself.
It was essential to know the order in which the changes were made…for example, the result will be different if you created some transactions and then adjusted the account balance, and if you adjusted the account balance and created the transactions after that. So, we’ve decided to give each change an incrementing number… We call it “command number”, because essentially changes and commands are the same thing.
It was also important to be able to tell which commands come from different devices. If we didn’t know that, we couldn’t possibly merge the data correctly (if changes were made on several devices at the same time). So we’ve decided to group the changes made by one device. Each group we call “revision”. MoneyWiz uploads and downloads data from the server every 20 seconds. All changes made within these 20 seconds go in one revision. If after another 20 seconds you’ve made more changes, they will go in a new revision and so on. Revisions are also incremental, so we can properly arrange them.
The process works like that…
When you first sign up for the Sync Everything! service, MoneyWiz will send commands for your entire database file… It’s pretty much like it will describe what you did so far, and save that informations to the server. Since all that information comes from just one device, it’s all grouped under one revision. This is revision number 1.
After that, every time you make some changes, MoneyWiz will upload them and create new revisions.
When we spot that there is a revision on the server that hasn’t been downloaded yet. MoneyWiz will download it. This would happen when you did something from another device.
Not only this solution proved to work beautifully, but it also brought another perk – we could revert data back to any previous revision… This way, if we made an unnecessary change or something just messed up with the data, we can always revert it back to the state we want. For now, this functionality is not available for the users, but we have it on our radar to add it in the future.
Security of Sync Everything!
So, what about security? Here’s what we’ve done:
- Your data is encrypted on the server, with industry standard AES encryption
- The entire server is backed up every day, and we keep the backups for the last 30 days
- Only you have access to your data, unless you provide your e-mail and password to someone else
- We use managed dedicated server, being a firewall and server administrators constantly make updates and routine checks to make sure the server is secure from the outside world.
- We’re in the process of obtaining several security verifications, such as Verify, TRUSTe and McFee Security.
Your data is not currently encrypted on the iOS device itself and the communication with the server is not via SSL yet. The reason is that to legally do this, a company needs an ERN number from the US government. We’re expecting our ERN number to be ready any moment now actually, so I expect to start encrypting the local database + make SSL communication in a month or two from now.
Future Development
We want to improve Sync Everything! even more, so we have a few things on our to-do list that will help us do that:
- Start compressing data and merging old revisions automatically, to speed up the initial sync. Currently the initial sync (the one that happens when you just (re)installed MoneyWiz and you sync from existing sync account) can be slow when you have many revisions. There are people with more than 5000 revisions and the initial sync can take 1-2 hours for them. When merged, the speed improves around 10 times (we’re already testing a solution, and that’s what our benchmarks show). We also have few other significant performance improvements in mind, which should be able to make the initial sync many many times faster.
- Ability to sync attached images – we don’t sync them now, because we can’t afford to keep the service free and pay the server costs if everybody decided to sync all their receipts. We do plan to give a free disk quota for images, and anything above it to be on a subscription plan via In-App Purchase. That is to be expected in major version 2.0 of MoneyWiz, which is planned for summer 2013.
- We have such a great service there… why use it for MoneyWiz only? We could develop other great software, that syncs automatically…
Share this:
Sounds like a good system. However, is there any way to keep a backup of the data ourselves? Having investigated the Moneywiz folder using iExplorer, I can’t find a copy of the database to keep a copy for myself (just in case).
I assume data is stored locally even with sync everything because I can still see all my items with no Internet connection. I guess it might be a bit clearer to do so when the Mac version is released? (Guessing using Time Machine would enable us to restore the local version to a previous state though if in sync, the sync would replace it)
Hi Chris,
For now, there is no option for manual backup, but we’ll add this later this year.
For the Mac, it will indeed be easier, because it has a file system you can actually access.
As for the data- yes, it is stored locally too.
Thanks
Another benefit to getting the Mac version then, on top of the already excellent service the iPad version provides. Hopefully the beta is progressing well and I’ll have fingers crossed Apple approve it soon after you submit it!
Hi, I love this app, it works really well and does what it says it does, well done to all of you. I just can’t wait till you add a debt section instead of a work around
Firstly, i do love your sync – it works effortlessly, quickly, and so far without error. Just what anyone wants from sync.
However, i am not convinced by your argument against Dropbox syncing. I’ll explain why be explaining how the “Day One” app syncs via Dropbox. Here goes:
I use the “Day One” app and sync via Dropbox. I have just tried what you described above by taking my iPhone and iPad offline. I then changed each device’s Day One data by entering new values. I then brought the devices back online and did a sync. The result? Both devices became perfectly in sync; they both pulled the changes the other device made.
So, it’s clearly not the case that apps syncing over Dropbox must sync the entire database. In fact, it’s all about how the device stores its data. In Day One’s case, it stores every “change” to the database as a single file on disk, meaning that the app syncs only the changes (reducing bandwidth requirements), plus allows apps to make changes offline and sync effectively once online.
Also, i use OmniFocus and it uses webdav to store it’s database files (giving me control over the webdav server i use), and again, i have never seen any issues of data loss of “entire database syncing”.
So, it’s not actually clear why your schema is any different. It’s also not clear why you really use your own data storage. For me, i’d very much prefer to have control over where my data is stored. Being forced to give you all of my data is for sure a real issue of using your apps. It just doesn’t feel right, although i do it because your apps are amazing.
Andy
Hi Andy,
webDAV is a technically possible solution, but in the same time it’s too geeky for 99% of people. One of the things we pride with is the simplicity of using MoneyWiz. WebDAV would take that away!
As for Dropbox – possibly someone did it as you describe… no idea, and frankly I don’t really care. What would happen if Dropbox died one day? We are left without sync? We don’t want such dependency. Also, going for Dropbox sync shows just one thing for the developer – you don’t have the balls to do something better! Well, we do think of the future of the company, and we see huge potential. But for this, we need to be able to control every aspect of our software. Pretty much like what Apple does, but of course on a much smaller scale.
P.S. Talking about security – Dropbox was hacked 1-2 years ago, and all of their customers were compromised…
Fair points. I do agree with your need for end to end control, plus your desire for simplicity. It all makes sense. Plus, i appreciate that you had to go with what you wanted to go with. It’s all about choice and you made yours, and it resulted in a great set of apps, with a stunning sync service.
That said, I would recommend reviewing this whole blog post though, because it seems partly based on drawbacks that don’t really exist. Why argue against Dropbox (saying how it demands you sync the entire database, and that you can’t work offline) when you haven’t tested them yourselves? Saying you “don’t care” that others succeeded in using Dropbox for this exact use is plain bizarre!
To me, it’s clear that what you’re really saying is that you didn’t want to try Dropbox because you had good business reasons not to want to depend on that service. That’s a good enough reason, without proposing that it was technically impossible.
Best Rs
Andy
Are you going to offer a ‘do not sync’ option on the Mac version?
Personally, I have no need to sync finance data between my macbook and other devices. I download everything from the bank and import it at the end of each month rather than enter every transaction manually and don’t need to push data out to other devices to keep track of spending.
I’d actually feel unhappy about storing my financial information in your cloud, but would rather store it encrypted on my Macbook.
Yes, just don’t create a sync account and it won’t sync.
Nice article
Hope MoneyWiz can set networkActivityIndicatorVisible in StatusBar, that can be a clearer understanding of its synchronization is working status.
Really looking forward to birth of Mac version.
Nice tip, thanks!
Added it to our to-do list. For now though, on the iPad you can see the settings wheel rotating every time it syncs. On the iPhone you need to be on the main screen though, to see it.
But the habit of many people is to look at the upper left corner. XD
Well congratulations, you guys really have something strong going on here. so far you guys have left very little to be desired. A very good thing. I wonder though what i can do to sync to have my ipad and iphone sync as it should. from the very first sync the transactions and their individual total sync fine but its the grand total of all the transactions from my checking account that do not match. in my ipad it show that i have thousands of dollars but its only my iphone that shows the correct amount. Whats up with that. I have even reinstalled the app on the ipad twice and they still dont match. Whats up with that?
Hi Luis. This is known bug in version 1.3.2. Please install 1.3.3 that was approved just few hours ago, and resync. It must be OK. If not drop us an email.
Without question this is right up there with the genuine five star apps and I’ve said so in my review. The sync is head and shoulders better than anything I’ve come across before. But I have to say the security side of things is one of those glass half full issues. On the one hand it’s brilliant that no data is stored on my iPhone 4S or ipad2. Clealy this is a better, more secure option as should you lose our device, disaster ! it wouldn’t take long for some geek to crack the local password and there are your finances to look at. But I have nightmares about you guys suddenly disappearing one day and when I go to login nada, zilch access and bang goes my eleven account family finance access and I have no backup…. It really is a worry but frankly I just have to trust as the competition is rubbish !
I do wish that you could improve the scheduled area of the app. I just can’t get it ight and I know you have looked at this but please can’t you at least give us the option to generate automatic scheduled transactions each month on a given day ? Instead we have to remember to go in and tap every one and there seems o be a bug in the way this works ie I can’t get the period thing to work and properly show that the next scheduled transaction to a particular account is due on say the 1st of May, it just shows the 1st of June…nd so on. Very frustrating ! I know one people prefer that they must go in and generate the scheduled transactions on thevirst of the month etc but hey, that’s great, we are different and have different needs and approaches and the option would be wonderful when you have many scheduled transactions going on across your accounts.
But…this is a superb vfm app and I love it, our smily finances are totally yours guys so Look after us !
Cheers, Mark
Hi Mark,
I want to clarify one thing – the data IS stored on the device as well…it’s on the device and on the sync server at the same time, otherwise it would be impossible to use it if you were offline.
As for the suggested features – don’t worry
They are on our to do list and will be done this summer.
Your apps are amazing: I switched to them few weeks ago when a well-known competitor announced that the cloud-based syncing of its applications was postponed to the future.
One concern: since installing your apps and enabling syncing, I noticed a decrease of battery life. Could it be because of the check that you do every 20 seconds? (even if the app is in background)
Hi there,
Thanks for the positive words
I doubt the decrease of battery life would be because of MoneyWiz, because it doesn’t sync while in background mode. So if you keep it in background mode, it’s pretty much the same if you have sync enabled or not.
You could try restarting your device… possibly some process has gone into a loop (happens from time to time). You can tell for sure something like that is happening if the device feels warmer than usual.
It’s good to know that MoneyWiz doesn’t sync every 20s while in background mode
I will investigate the actual cause of the battery life decrease…
I love this app. I have tried all of the others including money well but I had sync issues with that after one of their upgrades and there were other things that annoyed me. I run this on my IPhone and iPad and the easy syncing is a real plus. I look forward to getting the Mac version when available. It is one of my favorite apps… Cheers.
I just got your app tonight and I like it a lot. I have a couple of suggestions though
(1) An income budget would allow us to watch our total income on a monthly basis
(2) Please create the PC version of your app as soon as possible because I have a PC and I am sure a lot of your customers do too and I would love to have it on my computer as well as on my iphone and ipad!
(3) It sure would be nice to have the option of which report I would like to feature, rather than the one you have in that area.
(4) Please give us the option of creating our reports in the way we want it, not having them only as the type you chose. Personally, I dislike pie graphs!
This financial app will be one of the most important apps I will use and I can’t congratulate you enough in the creation of this one, it is really awesome and I think you for the enormous amount of work that you guys have already put in it. I know that the choices I have asked for and many of your customers have asked for will mean a lot of work for you guys but, I hope you will do them all because that will make your app the best one of all apps period!
Syncing devices directly, via WiFi or Bluetooth is too much hassle. It’s just and old and obsolete way of doing this. Add a third device, and you will be looking into a serious waste of time. Also, the devices need to be physically close to each other for this to work. So, this sort of syncing solution is off the table…at least for us.
That’s too bad – otherwise I’d give serious consideration to using your products. I don’t care how “cool” or “modern” cloud sync has become, I don’t trust *anyone* to have my financial data on some faraway server somewhere. Local sync is the only option I think any responsible person should consider.
Since you’re already building your own custom cloud sync solution, why would you not implement your own local sync solution as well? Considering how many other finance apps are ditching local sync for some kind of cloud sync, you could really help your product stand out from the crowd by providing a local sync option.
Spade, local sync is on our to do list, but for now it’s not decided when to be implemented. The reason is that very few people actually request it. Put in other words, there are much more requested features and only 24 hours per day…
Curiously asking, is there a way to create folders to group some accounts together? I travel a lot and have cash accounts for many different currencies that I would prefer to group under one folder. Besides that, this certainly is a fantastic app and I can’t wait till the MAC version is released so that I can switch over from iFinance to MoneyWiz!
Hi Mik,
Grouping accounts is not yet possible. We’ll be doing this for major version 2, which is planned for summer 2013 as a paid update.
This is an amazing app and I’m really enjoying it on my iPhone and iPad as well. You guys hit a homerun with this one. However, I seem to be experiencing an issue with sync at the moment. There are two transactions missing from April 22nd on my iPad. They both show up on my iPhone, which reflects my correct balance to my actual bank account. Today is the first time I’ve noticed it, which is odd since the missing transactions took place on 4/22/12. I have an account named “Joint Checking” and an account named “Groceries” and on 4/22/12 at 7:07 PM, I transferred $51.92 from “Groceries” to “Joint Checking” and then made a subsequent entry in the “Joint Checking” account within the same minute to show that the $51.92 had been spent as an Expense at Wal-Mart. Neither the transfer of dollars from one account to the other or the Expense in the “Joint Checking” account are showing up on my iPad. These are the only two transactions that haven’t reconciled for between the two devices. Thoughts on what might be wrong or how I can get the iPhone and iPad reconciled? Thanks!
I’m sorry, I also forgot to mention that I’m already using the latest version for iPad… 1.3.3. Thanks.
I went ahead and deleted those two transactions from the iPhone and then re-entered them on the iPad and now they are both reconciled. Hopefully it was just a one-time occurance. I’ll let you know if it continues to happen. Again, I think you have a great product! Thanks!
One more question… the Major Update you mentioned (v2) is a paid version or free to update for current user? is it for what about next Mac Version?
Hi Ben,
Major version 2.0 will be a paid update for all (iOS and Mac current users included). We’re hoping to release it in summer 2013, so it will be first paid update for all 3 apps, where the Mac would have been around for 1 year, iPhone 1.5 years and iPad 2 years. During this period of time, all updates will be free.
Now I feel I love your app twice.
.
I recently started to learn and develop for Mac OS X, today after a busy day with all my assignments and education matters reading this article was a great inspiration.
I too am reluctant to store my financial data on MoneyWiz server regardless. I have purchased the Mac app but still I am hesitant to use it because of syncing between Mac and iPad is only through Sync Everything.
Can you add the wi-fi sync option or even a syncing through iTunes would also help. This will ensure that my confidential data is on my own devices which I back up quite frequently and not to be in any kind doubt that it will be compromised anyway.
I am in Canada and I reject to use Mint.com (everything is on their server), MoneyWell (now offers only dropbox sync). I like the concept of MoneyWell (envelope budgeting) but I can’t use it unless they give us wi-fi or iTunes sync feature and release an iPad version.
I like MoneyWiz but internet/cloud/(whatever you want to call it) based syncing is a no-go for me.
Hi,
I do agree with Rehan (05-11-2012) … User should have an option to synchronize between devices via Wifi or to bakup (dump) via iTunes without having to use the online Synchr.
Hi Rehan,
Manual backup and restore will probably come as a feature at some point. That would work without using any online sync service.
Guys
Glad to see a mac version as I love the iPad version however I am surprised around your comments on Dropbox and not allowing me to save the file down and share it between my applications. It’s my choice if I want to use 600mb of data per month.
I am surprised that you are not syncing over SSL. I understand the formalities of applying for an ERN but you cannot say in one statement security is a priority then not use the industry standard SSL. Personally I would have held back until you got the ERN fully set up, but at least you’ve mentioned this on the blog post so people can decide.
I have already engaged with you about security but it would be great for you to become more transparent and explain more on how you handle the cryptographic keys, do you store our passwords, are they hashed as this is financial data we are talking about.
Whilst not at the top of your feature list until I see either local sync or a way to exchange my data between devices without your cloud I will not be purchasing the other products. It is my data and I want to be in control of where it is sent and sits. At least if I send an accounts file to my mac via email i control where it is sent. If I use Dropbox I can encrypt the file further. If I use local sync over wifi I control the network.
Even if you can offer Dropbox or local sync what about iTunes USB transfer or let me email the file out so I can get my data out then import it into the other application.
I will keep watching on the progress of this but for the moment it’s the iPad all the way for me.
Thanks
Craig
Hi Craig,
I am glad to say that we recently obtained our ERN, so SSL and hard-core encryption is coming soon
As for the algorithms – sorry… this is private information…but let’s say that SilverWiz, Apple and BIS agree on what we’ll be using and how, so it must be good
As for local sync – I will repeat myself – local backup/restore may come in the future, but it’s not high priority for us. It spoils the entire syncing experience, it’s old and obsolete, and 99% of our users are perfectly happy with Sync Everything. We’ll most of all add it for technical support reasons, not so much for sync, but local wifi sync would be one way to use it.
Hello
Just wanted to ask you a question, before I purchase the apps. I would like to be able do manage and keep in sync the information, between 2 Macs, 2 iPhones e 1 iPad. Can Moneywiz handle that situation? Also do I need to purchase a different license for each Mac (both use the same Mac Store account) and iPhone (same App Store account)?
Thank you
André
Hi André,
I can gladly confirm that MoneyWiz’s Sync Everything can handle such situations with ease. Our sync service is not affiliated with Apple’s iCloud, therefore these devices can use different AppStore/iTunes accounts and MoneyWiz will still sync, because you will have a totally separate account for Sync Everything.