I have learned some years ago that ETF's are strange and wonderful creatures. Simple, long-only mean-reverting strategies that work very well on ETF's, won't work on their component stocks. (Check out a nice collection of these strategies in Larry Connors' book "High Probability ETF Trading". He has also packaged these strategies into a single indicator, the ETF Power Ratings, on tradingmarkets.com.) Simple pair trading strategies like the one I discussed in my book, also work much more poorly on stocks than on ETF's. Why is that?

Well, one obvious reason is that, as Larry mentioned in his book, ETF's are not likely to go bankrupt (with the notable exception of the triple-leveraged ETF's, as I explained previously), because a whole sector or country is not likely to go bankrupt. So you can pretty much count on mean-reversion if you are on the long side.

Another obvious reason is that though there are news which will affect the valuation of a whole sector or country, these aren't as frequent or as devastating as news affecting individual stocks. And believe me, news is the biggest enemy of mean-reversion.

But finally, I believe that the capital weightings of the component stocks also play a part in promoting mean-reversion. Typically, weighting of a component stock increases with its market capitalization, though not necessarily linearly. Perhaps large-cap stocks are more prone to mean-reversion than small-cap stocks? But more intriguingly, can we not construct a basket of stocks, with custom-designed weightings, with the objective of optimizing its short-term mean-reversion property? I (and others before me) have done something similar in constructing a basket of stocks that cointegrate best with an index. Can we not construct a basket that is simply stationary (with perhaps a constant drift)?

Now, perhaps you will agree with me that ETF's are strange and wonderful creatures.

Let me talk about a topic that is far more mundane than the usual high-brow theoretical discussions of strategies and algorithms, but that has no less long-term impact on the bottom line: what is the best office environment for research and execution of quantitative trading strategies?

I have worked in different office environments before, so I feel qualified to offer an informed opinion.

At Morgan Stanley, I huddled over a desk that is semi-partitioned from the rest of the office: nobody could see or bother me unless I or they stood up. At Credit Suisse, I shared an office with 2 other prop trading colleagues, one of whom was prone to freely sharing his opinion on various current affairs with his officemates. (On the other hand, he complained my biting an apple for lunch was too loud for him.) At Maple, a hedge fund in New Jersey, I shared an office with about 100 other colleagues on the trading floor, many of whom were prone to same opinion-sharing temptation.

Here at my own firm, I sit in solitude (except for my cat) in my basement office, my beloved classical FM streaming over the internet, my desktop electronically connected to my partner in our Chicago office, our trading servers at Amazon and elsewhere, and other clients and partners around the world, but unmolested throughout the day unless I voluntarily pick up the phone or answer an email or instant message.

Can you guess which environment is the one I find the most productive? Which one has the least stress? And which one contributes most to the bottom line of my employers/partners/clients?

(Hint 1: read Timothy Ferriss' book The 4-Hour Workweek. This guy checks his email only once a week.)

(Hint 2: my trading Sharpe ratio went from negative to >7.)

P.S. I look forward to meeting some of you at the Automated Trading Conference in London this Friday (my talk will start at 0900), and others at my pairs trading workshop on the preceding two days.

I confess I don't know much about flash orders, not being one of the Big Boys on the Street, until I read that the SEC is banning them. (For a clear diagrammatic explanation of flash orders, see here. For a refutation of some of the myths and misunderstanding surrounding flash orders, see here.)

It seems to me that flash orders can be understood as "request for liquidity" issued to various potential market makers/liquidity providers, not unlike the usual "request for quotes" (RFQ) common in other industries. They are issued when there is not enough liquidity on a specific exchange to satisfy an investor's need, and they ultimately benefit investors by lowering their transaction costs. The fact that high frequency traders are able to make lots of money by providing this liquidity is besides the point. Liquidity providers are supposed to make money by providing liquidity!

Some people, including Senator Charles Schumer and this New York Times op-ed, believe that flash orders are akin to front-running, a clearly illegal trading activity. But they are wrong. Front-running means that if you know someone is going buy a stock, you step in front of them
and buy it cheaply first, hoping to sell it to this slower buyer at a higher price. In the case of flash orders, the high frequency traders are instead selling this stock to the original investor, often at a lower price than available elsewhere and thus benefiting this investor, hoping that the prices will come down in the future after this liquidity need subsides. This is manifestly not illegal. This is what a market is built for!

Another way to understand that flash orders are not at all front running is that anybody, including you and me, are free to put in limit orders at the same price as those of the high frequency traders, way ahead of time, in a specific exchange, and become liquidity providers ourselves. You don't have to wait for a "request for liquidity" before doing so. And presumably you will reap the same benefits as the high frequency traders. You are not taking any additional risks over the HF traders either, since if no requests for liquidity ultimately arrive, you are not any worse off for wear. You cannot begrudge the profits of the HF traders just because you didn't put the limit orders in place beforehand!

Maybe there are some other angles which I miss which can convince me that flash orders are evil. But until my kind readers convince me otherwise in the comments section, I will regard this piece of legislation as another SEC attempt at demagoguery.

It occurs to me that the only way in which a trader can become more than a completely selfish, self-enriching, narcissistic person is to trade well enough so that you can manage other people's money and thus saving these investors from crooks and charlatans (provided you are convinced you are not a crook and charlatan yourself).

Other traders have advanced other arguments in favor of trading. But I am not convinced by them.

They say that we provide liquidity to other long-term investors who may need to liquidate their investments. But then, this applies only to mean-reversal strategies. Momentum strategies take away liquidity from the market, and in some cases exacerbating price bubbles. Certainly not something your grandma would approve.

Others argue that momentum strategies help disseminate information about companies through quick price movements. But can't we just watch Bloomberg or CNBC? Do we really need some devious insiders to convey that information to the rest of us through price movements?

No, I think that independent trading should serve only one purpose (besides short-term self-sustenance): as training and preparation to become a fund manager. Once you graduated from independent trading, you then enter into the grand contest among all fund managers to see who can best serve and protect investors' assets, (and be rewarded according to your standing in this contest.)

I know, this is the idealistic way to look at things. Serving and protecting seem to be what policemen should be doing, not traders. But as in quantitative trading, I think it helps one becomes more successful in one's activities by having a simple guiding principle or model. And it doesn't hurt that in this case, the principle would also be conscience-nourishing!

Author Malcolm Gladwell, in his fascinating bestseller "Outliers: The Story of Success", cites neurological research showing that "10,000 hours of practice is required to achieve the level of mastery associated with being a world-class expert." This seems to apply across many different types of experts, whether they are "writers, ice skaters, concert pianists, chess players ... Even Mozart ... couldn't hit his stride until he had his ten thousand hours in".

Reflecting on my own experience, I have become consistently profitable only after 4 years of actual trading (research alone doesn't count -- real money need to be at risk.) So while the number of hours may not be exactly 10,000, the order of magnitude is about right.

So if your trading has not been profitable, ask yourself this: "Have I traded 10,000 hours yet?"

Paul Teetor, who guest-blogged here about seasonal spreads, recently wrote an article about how to test for cointegration using R. Readers who don't want to pay for a copy of Matlab should find this free alternative with similar syntax quite interesting.

I have given a 2-part interview (here and here) on the various nuances of backtesting on tradingmarkets.com. Most of the ideas have been covered in my book, but it does serve as a summary of what I consider to be the most important issues.

For those of you who are interested, I may be giving a workshop on general techniques in backtesting in London as well, in addition to my pairs trading workshop. Additional details will be available on epchan.com at a later date.

Triple leveraged ETFs marketed by Direxion have been all the rage lately. The fund management company says that they do not recommend buying and holding these ETFs. But is there any mathematical justification for this caution?

Before I answer this, it is interesting to note that these ETFs (e.g. BGU is 3x Russell 1000, TNA is 3x Russell 2000) are managed as constant rebalanced portfolios, a concept I discussed before. In other words, the fund manager has to sell stocks (or futures) when there is a loss, and buy stocks (or futures) when there is a gain in the market value of the portfolio, in order to maintain a constant leverage ratio of 3. This is also identical to what Kelly formula would prescribe, a methodology discussed extensively in my book, if the optimal leverage f were indeed 3.

However, the optimal f for such market indices are quite a bit lower than 3. Both Russell 1000 and 2000 have f at about 1.8. This means that since the funds are leveraged at 3, there is a real possibility that sustained losses could ruin the funds (i.e. NAV going to zero unless new capital is injected, which, er..., reminds me of a Ponzi scheme). So I would argue that not only should an investor not hold these funds for the long term, the funds themselves should not be leveraged at this level. Otherwise, it is a disaster waiting to happen.

For readers who do not want to pay for a commercial Matlab2IB API, Max Dama has put together a free alternative. Domenic has provided some additional sample Matlab codes for trading.

A user of the commercial product that I previously mentioned reports that "My problem with the matlab2ib product was that it did not have a function for all the Active X methods. ( for example the Market Scanners, Real time Bars and Fundamental Data methods are missing). I also had issues when I tried to steam in trades data(I'm not sure if the matlab2ib product allows you to even do this?)." Apparently Max's API has included these methods, though I have not personally tried them.

You can find an interview of me in the July 2009 issue of Technical Analysis of Stocks & Commodities magazine. I mentioned in that interview and also in my book that I believe stop loss should only be applied to momentum strategies but not to mean-reverting strategies. I explained my reasoning better in my book than in the interview, and so I will paraphrase the explanation here.

In algorithmic trading, it is reasonable and intuitive that we should always make use of the latest information in determining whether we should enter into a position, whether that information is price, news, or some analysis. Let's call this the Principle of Latest Information. (If someone can think of a better or sexier name, let me know!)

So let's say we have a stock model based on price momentum, and we entered into a long position based on a recent positive return on price. A few minutes later, the price went down instead of up, causing a big loss on our position. If we now ran this momentum model again, very likely it would tell us to short the stock instead because of the recent negative return on price. If we did that, we would be exiting the previously long position and became flat. This is in effect a stop loss, and it follows strictly from adhering to our model and our Principle of Latest Information.

In contrast, suppose we now have a stock model based on mean-reversion, and we entered into a long position based on a recent drop in price. A few minutes later, the price went down further instead of up, again causing a big loss on our position. If we now ran this mean-reversion model again, it would definitely tell us to buy the stock again because of the ever cheaper price. The model would not ask you to exit this position and take a loss. Hence, adhering to the model and the Principle of Latest Information will not lead to a stop loss for a mean-reverting model.

(Now, if we hold this losing long position long enough, the model will incorporate new historical prices into determining its long or short signals as it retrain itself, as the Principle of Latest Information says it should! At that time, it may indeed recommend that we exit the previously held long position at a loss. But this adjustment takes place at a much longer time scale, and therefore cannot really be considered a stop-loss in its usual sense.)

More generally, I find that at every turn, and not only in the realm of stock trading, applying the Principle of Latest Information always help me to be disciplined and not be afraid to enter into new positions, take loss or endure a drawdown as the case may be.

Larry Connors and Cesar Alvarez (the guys behind tradingmarkets.com) recently published Short Term Trading Strategies That Work, a nice collection of simple technical trading strategies that you can easily backtest and verify.

As I have argued in my own book, simple strategies are often the ones that work best. As with any published strategies, you may find that their backtest performance may not be as high as advertised if you test them on a different time period or a different security, or with different transaction cost assumptions; but the main value of these strategies is that they serve as an inspiration to trigger your own imagination and motivate you to refine them further.

(For e.g., though the book mainly covers long-only strategies, you can easily imagine the accompanying short strategies.)

To be quite honest, this is one of the few books on trading strategies that I actually manage to finish reading from cover to cover.

Alphacet told me that they have a job opening for a quant who will be helping their clients backtest trading strategies, among other responsibilities. Given that Alphacet's clients include several major investment banks and hedge funds, this position should provide pretty good close-up view of how the major quantitative players operate.

I just published an article "MATLAB as an Automated Execution System". (It is available to readers of my book and subscribers to my Premium Content website.) It comes with example MATLAB codes executing a simple Bollinger-band high-frequency E-mini trading strategy.

As I mentioned before, I now find MATLAB to be a good platform not just for backtesting, but for automated execution as well. Of course, not all brokerages have API's that connect to MATLAB. My example codes are for submitting orders automatically to an Interactive Brokers account.

In general, I find that writing execution programs in MATLAB is a breeze compared to C++, Java or even C#. It takes about 1/5 the development time of a C++ program. Any performance limitations will probably not be due to MATLAB, but to the latency of your brokerage in updating positions and order status.

I will be holding a 2-day, hands-on, pairs trading workshop in London, October 14-15. It will be held in conjunction with the Automated Trading 2009 conference organized by the Technical Analyst magazine. Please see details here.

In my book, I mentioned 2 seasonal trades in natural gas and gasoline futures that have been consistently profitable for 14 years. (Mentioned here and here also.) And not only in backtest: I paper-traded them in 2006, and actually traded them in 2007-8, and all 3 years were profitable. How did they fare in this recession year? Quite poorly.

Depending on your exact entry and exit points, the gasoline trade lost about $2,500 per contract of RB. The natural gas trade lost about $7,700 per contract of NG.

You may have heard that natural gas price is at a 6-year low. In fact, we are not seeing any increase in industrial demand for natural gas. Apparently, somebody has forgotten to tell the nation's industrialists that an economic recovery is supposed to be under way.

Will I enter into these seasonal trades again next year? You bet I will.

As an algorithmic trader, I am constantly in search of a better physical infrastructure where I can connect via the internet to my execution broker at the highest speed and with the least possibility of outage, and at a reasonable cost.

To that end, I would like to mention Fios, a fiber-optics service from Verizon with download speed of 50 Mpbs, upload speed is 20 Mbps, both faster than your typical T-1 line (1.5 Mbps). Furthermore, it costs only $45/month. Hey, even Paul Krugman has installed it at his home!

(I haven't tried it myself, and would like to hear from those of you who have and see if it is time to say goodbye to T-1.)

And as I have reported earlier, I am also constantly looking for a good cloud computing platform so that I can run more strategies without cluttering my office with computers. Finding one will obviate the need for any big investment in internet connectivity at the office.

To that end, I have been trying out Amazon's EC2 for several months. I use it to run one of our strateiges, and I have to report that my experience is mixed.

Firstly, if you are not an IT person, it does take a lot of time (8 person-hours?) to get set up and running, especially with their securities precautions. The learning curve is steep.

Secondly, and more annoyingly, the instances sometimes fail to start properly, or fail to bundle properly. (Bundling means saving the software configuration for future use.) I am using Windows instances. Maybe those who use Linux instances have better experiences?

Thirdly, and most annoyingly, when a new instance is started, Windows often cannot automatically synchronize its clock with time.windows.com or any other internet clock. As a result, the time is often wrong. Now, this may not be a big deal for usual office work. But when your automated trading strategy depends crucially on the time of the day, it can be quite fatal to your profit. If anyone has experienced a similar problem with Window's clock and know a fix, please let me know!

Despite all these hassles, I am still running strategies on EC2, hoping that once EC2 get past the beta release, things will be better.

In a recent post (hat tip: Russell M.), Tyler Durden at Zero Hedge quoted a quant trader saying that "Anyone who is doing anything sensible right now is either losing money or is out of the market entirely", and that "liquidity deleveraging is approaching (if not already is at) critical levels", and finally the scariest part: "we have crossed into major statistically deviant territory, likely approaching a level that is 6 standard deviation away from the recent norms."

He pointed out that NYSE weekly volume is running about 9% below 52 wk average. But this may not necessarily be the result of deliberate hedge fund deleveraging or increasing risk-aversion by quant traders. From my personal experience, the usual opportunities for mean-reversion have just markedly decreased in the last few months, with much of the cash sitting on the sideline. I believe that quant traders are still ready jump in at any time to provide liquidity should the market demands it. I don't think that the recent market condition portends a 6-sigma event, but if one should occur, it may actually be a great profit opportunity for many short-term mean-reversion traders just as in those past 6-sigma events.

Thoughtful comments from a reader John S. from the UK on his experience with trading technology and models:

"I have been developing my own personal automatic trading systems using Excel VBA and based on rules I have developed over the years as an active private trader investor using both technical and fundamental data analysis.

One of the key merits in adopting an automatic trading system approach that has helped me is to avoid the temptation for manual interference and thereby improving profitability by maintaining consistency. I have found the challenge of developing a successful system very rewarding from a personal perspective as I recognise that there are many that have tried and failed. However one problem I have encountered is my ongoing desire to regularly modify and improve the system which I have found can become counter productive as there is a real danger that system development becomes an end in itself! I just can't seem to stop tinkering as soon as I come up with a new idea or feature!

One advantage of using Excel VBA that I have found is that it is inherently flexible as it facilitates the processing of data which can be important especially when using fundamental data as part of the system. In this respect I recognise that every trader is trying to build in an edge that will make the system more profitable. I have noticed that many traders seem to only focus on price by trying to seek an edge by looking at special indicators or combination of indicators etc. Combining price data analysis with a Factor Model approach is a challenge which is ideally suited Excel VBA as it can be easily used to process both fundamental and macroeconomic data into a form that can be integrated with price data analysis.

I recognise from your book that Matlab is more powerful than Excel VBA and may be just as flexible in integrating fundamental and macroeconomic data but I just wanted to draw your attention to benefits I have found using Excel VBA which may suit those who like myself are more comfortable in using Excel VBA and are reluctant to change. Other features that can be exploited that I have found helpful when back testing are automatically producing Price Charts that incorporate Entry and Exit points which provides visual reassurance that the system is working as intended as well as generating automatic Word reports recording key output for future reference.

I am sorry if I sound too much like an advert for Microsoft!"

As I mentioned in various previous blog posts, (e.g. see here), I believe mean-reversion strategies have been performing very well in the last year. Now here is an article (hat tip: Laurence) that provides concrete analysis to support this hypothesis. In fact, the author points out that most of the mean-reversion in recent years comes from the overnight close-to-open reversal.

Here is a new low-cost service called Alerts4All that offers technical trading signals for retail investors. You can, for example, have an alert sent to you every time a "Double bottom" pattern occurs.

A much more advanced version of the service will be rolled out soon -- I saw a demo today where you can backtest your strategies online, combining different fundamental and/or technical variables as entry or exit signals. They also have some built-in models for you to adapt (e.g. a model based on The Little Book that Beats the Market by Joel Greenblatt.) More interestingly, you can look at other people's trading models and their historical and/or real-time performance.

Matlab or Alphacet it is not, but I think it will be quite useful for many retail traders. It might even be useful to professional traders who want a quick-and-dirty way to test out ideas.