Information is power. Unique information sourced from undiscovered territory then is a new level of domination.

In the past following the markets used to mean gathering data from a few main sources: reading the Wall Street Journal daily, reading financial statements and SEC filings and locating patterns in prices. Later the Internet made it easy to find an enormous volume of financial data for free at the click of a button.

Today we are living in the era of the data revolution, where nearly every move we make can be digitized, tracked, and analyzed and every company is now a data company. Using all this un-mined information with the potential to be leveraged for commercial value in the financial industry is called alternative data.

It is estimated that asset managers are currently spending $2-3 billion on alternative data, including acquiring datasets, implementing big data technology, and hiring talent. Spending is projected to grow 10-20% annually, in line with big data growth in other industries.

Alternative Data and the Next generation of Algorithmic Trading

Algorithmic trading has been used in stock, futures and other financial markets by institutional and individual investors with many success stories. Algorithmic trading works by “feeding” the trading system with price data from the exchange market, data from news companies or even data from the web (for deep analysis such as sentiment analysis). While this unique market data is being collected, the system performs complicated analysis to look for profitable opportunities and acts upon those opportunities without human intervention needed. Algorithmic traders are constantly searching for trading signals in order to acquire investing advantages.

Currently, the world is creating 2.5 quintillion bytes of data daily. Feeding this amount of data to investment and trading algorithms represents a unique opportunity for Quantitative investors. The more data available to the algorithm the more “unmistakable” the result. However, finding an edge doesn’t depend only on the quantity of data quants have, but also on the ability to make use of data overseen by others and combine the ones that otherwise seem unrelated. In other words finding and leveraging alternative data.



Sources and applications of Alternative Data
Drones
Drones have become increasingly sophisticated in collecting data either by using sensors or by collecting images — from the number of cars in various parking lots, to the change in oil reserve levels in large storage tanks and from crop and forest health to digital management of excavation and construction sites — data collected by drones can shed light on the health of various aspects of the economy.

Drones are cost effective: they are able to capture large amount of data in a very short time and in non-ideal conditions, such as through rain, clouds, darkness, or even concrete walls. They can travel in sites that are hard to reach and are dangerous for humans to inspect. Farming, construction, Oil, retail, manufacturing and insurance industries are expected to be revolutionized by the vast amount of data drones and satellites are gathering, analyzing and modeling.

Web, app, and social media data 
When it comes to predicting consumer behavior data from the web, applications and social media are the most useful. Moreover, as companies move their business operations to the Internet, new data trails are being created that provide unique actionable insights on these companies, providing systematic investors with valuable new information and critical data points that others might miss.

Broadly speaking, brand mentions and trending topics can signal price movement. More examples of data that can be aggregated from the web and that provide interesting insights to investors are: headcount and hiring rates of a company, employee sentiment, online store growth and overlay competitors together with their demographic data, data on online product pricing and promotional activity, pricing by individual property and city, growth on a tenant-by-tenant basis, etc.

Geo-location
Smartphones are equipped with location services that allow us to use maps or weather functions, but also let mobile carriers know where we are at any time. That untapped data can be valuable to see what shops, hotels or restaurants individuals are visiting, a gold mine for hedge funds looking for clues on consumer trends.  Changes in foot traffic can indicate future changes in revenue and profitability for retail and restaurant locations.

Foursquare has been able to accurately predict footfall, and therefore sales, at Apple and Chipotle. Twitter, meanwhile, sells data to banks and hedge funds directly. The services offered by Foursquare and Twitter are evidence of the growing use of "alternative data" on Wall Street. These alternative data sets are also helping bring transparency to opaque markets like China, where economic data has known to be inflated.

Credit card tracking
The most valuable seam of data for investors is information that shows directly what consumers are spending their money on. Although it only offers a partial view of sales trends, combined with other data sets credit card tracking can offer vital insights. For example, combined with satellite images that can scan car parks and geo-location data from mobile phones credit card usage can show how many people are visiting various stores, the investment group can get a real-time idea of how companies are doing, long before their results are released.

On the level of a company data on transactions can give great insight into the Business Health Metrics. Tracking company payment patterns such as amounts, delays and delinquencies can help investors discern whether a company is paying its debts on time. Sudden increases in late payments can be a sign of weakness and cash-flow problems. This data, therefore, can help gauge debtor distress — a leading indicator of stock under performance.

Business Operations Data
Big data can allow investors to get information on the health of various companies, but can also support in increasing efficiency. After all, most of the new business value will arise from optimizing operations. For example, in factories, sensor data used to predict when equipment is wearing down or needs repair can reduce maintenance costs by as much as 40 percent and cut unplanned downtime in half. Inventory management could change radically, as well.

Cameras can measure the number of components along production lines, and an inventory-management system automatically places supply orders to refill the containers. In mining, self-driving vehicles promise to raise productivity by 25 percent and output by 5 percent or more. They could also cut health and safety costs as much as 20 percent by reducing the number of workplace accidents.

Insurance Data

Every company needs insurance of one sort or another, which means that the insurance industry produces a ton of data that’s waiting to be analyzed. The insurance industry could suggest signals about the automotive industry, fuel supply, and even route points if you include in-car transponders designed to bring down premiums. For example, the number of insurance policies that are issued for new cars, almost perfectly correlates with new car sales volume. This means that auto insurance data can provide investors a daily count of new car insurance policies, sliced by manufacturer. This data can then be used to estimate total daily new car sales per manufacturer. Anyone trading auto equities would be interested in this kind of data.

Internet of Things

Research reveals that companies currently under-utilize most of the IoT data they collect. When physical assets equipped with sensors give an information system the ability to capture, communicate, and process data they create game-changing opportunities: production efficiency, distribution, and innovation all stand to benefit immensely. Research suggests that the operational efficiency and greater market reach IoT affords will create substantial value in many industries.

IoT systems can also take the guesswork out of product development by gathering data about how products (including capital goods) function, as well as how they are actually used. Using data from equipment rather than information from customer focus groups or surveys, manufacturers will be able to modify designs so that new models perform better and to learn what features and functionality aren’t used and should therefore be eliminated or redesigned.

Opportunities and Risks


We’ve barely scratched the surface of potential alternative data sources. The data sources that will become available in the next decade dwarf what we have access to right now and companies will evolve to make use of all this data. Every single industry is going to be transformed by data and early access to that data means an inside view on all those industries and companies. However, there are still unsolved issues and complications related to alternative data.

Processing the data

The problem with all of this data is that there is no way a human can process the enormous amount of financial information generated every day. This is where artificial intelligence (AI) can help by collecting, analyzing, and modeling the data far more efficiently than a room full of human analysts. AI can quickly find patterns within enormous data sets. Using the tools of AI and machine learning, investors, analysts, and wealth managers can develop far more complex forecasts than ever before to increase their rate of return on investment. As the amount of available data continues to exponentially increase, the need for technology capable of sifting through the data becomes increasingly necessary for investment decisions.

Privacy Issues

The usefulness of this type of data for investment firms does not lie in the actions or details of single individuals; instead, firms are looking for broad demographic trends that are based on the aggregate of the data sets. However, there are still concerns regarding the privacy and confidentiality of personal information gathered from alternative data sources.

For the most part, data gatherers typically obtain permission to sell data, often via the terms and conditions users are often required to agree regarding the use of the information. However, this alone may not be sufficient to protect against privacy intrusions.

Alternative data will eventually become a basic aspect of portfolio management; for now, funds and other institutions should approach the use of such data with caution and should ensure that they are in compliance with new regulations such as the GDPR which will set the tone for data privacy regulation in the near future.



Edge or commonplace?


The landscape of data is constantly changing and analysts have to evolve to keep up with it. However, every model has a shelf life. Rare, unique and proprietary data eventually diffuses and becomes commonplace, easily available, edgeless data. Newly discovered datasets are rare and valuable because they contain alpha. But as time goes by, this data diffuses to a wider audience and its alpha content diminishes. Finally, the data becomes old and obsolete. Smart analysts who have internalized the spectrum of diffusion are aware of this dynamic and are always ready to adapt.

To conclude, the wave of Alternative data will only be alternative for so long, eventually becoming a core part of any portfolio manager’s toolkit. Eventually, everybody will have access to satellite data, just like everybody has access to fundamentals data today, and it will no longer be edge-worthy. But for now, unorthodox alternative data sources like logistics and consumer data are moving the markets. Those who can access the information the fastest will stay ahead of the market. To stay competitive, firms need to consume more data both traditional and non-traditional.

However, conducting research with alternative data does not always come easily; it often arrives in messy formats and can be difficult to handle. Treasures are often hard to dig out.Therefore, innovation in the future will consist in, “turning the data into knowledge” using machine learning and artificial intelligence. Creating the next generation trading algorithm that can analyze the vast amount of alternative data produced every single day, an algorithm that will be able to make deeply informed decisions and discover profit opportunities without the least human intervention.

A great future to imagine, but one that will belong only to those who are acting now:acquiring datasets now, implementing big data technology https://quantsavvy.worldsecuresystems.com/blog/quants-are-using-alternative-data