It’s understandable why you’d want to build an iPhone app.
With more than two million currently available, and continued growth of the iPhone’s market share, there’s still plenty of opportunity for you to capitalize on the opportunity.
As the number of innovative ways to solve problems continues to explode with these applications, iPhone users are becoming much more reliant on them as a means to manage their lives.
The customers are there. The opportunity is there. And your idea is there.
However, what will make all the difference in how well your app performs, and ultimately how well it’s received, will be the data that powers it.
So, in addition to building the actual “guts” of your application, it’s vital to pay special attention to all the information inside of it that will make it valuable.
And to do that effectively, you’ll have to be cautious determining where you’ll obtain the data that makes your app sing.
Why your data needs to be high quality
Data is the lifeblood of your application. It is what makes it useful for the people who’ll be using it.
Not only can the data you provide help differentiate you from the other applications competing for your users’ time and attention, but the more they use it, the more they will rely on the information provided to them.
And because your customers can make decisions with data within your product, you want to ensure they are making selections with the right inputs.
Therefore, the quality of the information you’re providing needs to be trustworthy and reliable. Otherwise, you’ll lose credibility with your users, and the reputation of your app will go out the window … and that’s not good!
3 ways to source data for your iPhone app
There are several ways you can obtain the data you need to power your application. Each has its own pros and cons, so you’ll need to carefully think about your current situation, capabilities and priorities to help you determine which approach is right for you.
1. Utilize user-generated data
There are a number of applications that are valuable to the people who use them on a regular basis, because of the way in which the app presents their own data.
Here the emphasis, would be in collecting and organizing the information in an effective way that enables the user to track progress or make decisions.
As more people use your app over time, you can take that aggregated data from all of them to generate statistics and identify other helpful trends that makes the application even more useful for your user.
Your data gets more robust and valuable over time, as more people use your application.
And because the people who use your app are the ones the data is coming from, you don’t have to invest additional resources to find data.
You’ve got to define your app carefully enough to where it can simply capture the relevant inputs from your users. Ideally, you want to minimize the amount of work they have to do, and prevent as much user error as possible.
In addition, the value of the aggregate data increases with more inputs. So in the beginning, with early adopters, there may not be enough information for them to compare their results to, to make the information worthwhile for them.
2. Partner with someone who already has the data
There are some suppliers who already have the information you need to power your app. If they’ve already done the work to get a robust and reliable data set for you, partnering up with them to license their data for your own use could be a win for all involved.
You get quick access to the information you need. And if the supplier already has an established reputation in the market, acknowledging that you use data from them can be an added source of credibility for you.
You are at the mercy of the other company. If they go out of business, you will need to scramble to find new data. They can raise their prices, and even include stipulations in the agreement as to how their data can be used.
Also, companies like this are often sourcing their data on their own to meet their own objectives. As such, the data isn’t specifically designed for your needs which may cause you to have to compromise on a few features of it.
3. Scrape the data from the web
If the data you are looking for is already on the web, you can harvest it from the websites where it resides.
Spiders are built to crawl the websites that have the information you want, and they then collect and aggregate the information for you.
You’re able to gather data from a variety of different sources relatively quickly. In addition, you would be in control of the data that you source, and wouldn’t have to share any sensitive business processes with a third party.
The information you want may not be available on the web. Also, the process of scraping data requires a certain infrastructure to collect and manage their information over time.
There are three primary routes you can take to scrape the data for your app.
Build the spider yourself
If you’re feeling ambitious, which, of course, you are because you’re building an iPhone app, you could build the data scraping spider yourself to capture the data for you.
This is an excellent option if writing code is a big part of your life, and you are more than comfortable whipping the needed commands together.
The main benefit of the DIY route for building the code to source your own data: You are in control of the process from start to finish.
You can design everything to function exactly the way you want, making the necessary changes without having to worry about anyone’s schedule other than your own!
Also, if you maintain data-capture in-house, it’s one less expense associated with building and running your app.
It can be risky, especially if coding isn’t your area of expertise. There are also a number of considerations associated with making sure the data is accurate and then properly integrating it into your app..
It’s time consuming. There are other tasks associated with building your app; therefore, focusing a significant amount of time on coding to obtain your data in good working order will decrease your available time to focus on other parts of running your app business.
Use ‘off the shelf’ scraping software
If you like the idea of remaining involved with the process of receiving the needed data, but aren’t quite so keen on writing the code yourself, there’s another option: Use ‘off the shelf’ scraping software.
There are companies who’ve created software having done the work to enable you to capture the data you want from various websites. All you have to do is pay a fee for the various APIs you create. Here are a few:
If you are familiar and comfortable with coding, you can take the data and integrate it yourself into your app. And if you’re not code-savvy, certain programs will help with the integration part, too.
In some instances, the applications make it as easy as going to a site and pointing and clicking which data you want, and their software goes to work behind the scenes to capture what you want.
It is a cost effective solution for harvesting data from simple web sites. You can save a tremendous amount of time by not building the code yourself. And if rolling up your sleeves and being involved is your thing, that’s possible as well. Once you receive your data set, begin to integrate it into your application yourself.
Off the shelf software is not designed specifically for your unique data needs, a certain degree of flexibility may be lost in what you’re able to do.
Also, it isn’t always easy to capture data from more complex websites, so ‘off the shelf’ software may not work as smoothly.
Are you familiar with concepts such as xPath and CSS Selectors? If not, then you’ll have to get familiar with them to use this method. There can be a steep learning curve to figure out how to use these programs if you’re not already familiar with some of the lingo and how it works.
Hire a third party professional to source the information for you
This is perhaps the easiest route to take. Especially, if your expertise lies more on the idea and business part of the app business, rather than the code building part.
Here’s how it works….
There are a number of businesses that are experts at building crawlers and spiders and other types of code that goes out and harvest data from the internet.
They live and breathe this stuff. They probably dream about it, too!
Several key players in this growing industry offer what you need at various price points. Begin by researching them and providing your exact specifications of what you need, who you need it from and how frequently the data should arrive to you.
Then they’ll custom engineer a spider to harvest your required data from your target websites. The spiders are run from behind a wall of anonymous IP addresses, so that the target website can’t trace their true origin. Once the spiders have completed their crawl, the harvested data is cleaned up, checked for quality, and delivered to you.
You don’t have to write any code. You don’t have to think about servers, storage back-ups, deduping, python, or other such technical stuff.
All that should put you at ease, if you’d prefer to spend your time thinking about design or how to entice people to download your app or making sure they use it on a regular basis.
After the initial process of selecting a vendor and giving them the specs of what information you need and when, you can go off and focus on the things you do best.
A good data partner will also keep you updated with the progress of your projects progress and discuss any tweaks and adjustments needed to keep your spiders crawling.
Also, if the data you’re sourcing to populate your app is mission critical to the value and effectiveness of it, then you’ll want to make sure the data is high quality.
Think of it this way: If you needed heart surgery, would you want the guy who’s read a few books, studied YouTube videos and watched a few episodes of “Grey’s Anatomy” to do the job, or would you want a trained heart surgeon to cut your chest open and expertly perform surgery?
You’ll need to relinquish control of a very important task to someone else outside your company. That can be really difficult, especially if you like to keep close watch and have your hands involved in all aspects of your company.
In addition, there’s also the investment associated with it. Hiring someone to harvest the data for you is definitely more expensive in pure dollars than doing it yourself. The pricing can range widely depending on the vendor, the quality of their crawling systems and the complexity of your request.
So, if you’re on zero or an extremely limited budget, this may be a sticking point for you.
Don’t let data sourcing be an afterthought
There are numerous things to consider as you work to build your app with many moving parts, from idea to development to commercialization.
However, sourcing the data that will power your innovation is one essential aspect that will require your close ongoing attention as your app takes off.
As Bill Gates, founder of Microsoft once said: “How you gather, manage and use information will determine whether you win or lose”.
Carefully review the three options listed above, evaluate your current circumstances and then explore one that seems like it’d work the best for you.