MVAR Hack 7.2

Screenshots

Description

Helps you find and test your ideas of what might be "leading indicators" of particular stocks or ETFs. For one day ahead indicators only. Explore "The Machine" of the market, and backtest your ideas forthwith.

For these types of simple models, if a backtest achieves 60%+ directional accuracy over a long period, it is considered fairly good. Our backtests have metrics which check how likely it is to achieve the model's level of accuracy by flipping coins for the same number of backtest days. For example, getting 6 out of 10 correct by flipping coins (60%) is much more likely than getting 60 out of 100 (also 60%) correct by flipping coins. You must look at % correct, and also how long you get this level of accuracy. The run length Plots give an idea if the model is getting better or worse over time.

We don't even report out model Fit quality to avoid confusion, since Forecast quality is usually worse than Fit quality. Hence: Always backtest to estimate Forecast quality.

The app models the (close-open) price direction (that is, today's price travel) of a given target stock as a function of prior days opening and closing prices of related stocks. Also works for ETFs if they are available in Google Finance historical data.

Check out our newest features: low-contributor variable weed-out for K-nearest classifier models, to reduce overfitting and possibly improve forecast quality (searching as always for the most parsimonious model, "as simple as possible but no simpler..."), and the use of volatilities as candidate leading indicators.

Some use cases:

- You want to keep a position open for 1 trading day only. Models built with this app can give an estimate as to whether the stock/ETF will go up or down today.

- You want to buy or sell a given stock/ETF for other reasons. Models built with this app can give an estimate whether Today is a good day to buy or sell, or whether you might want to wait until a later day to make a trade.

Some thoughts on backtesting, to assist with judgement calls:

http://www.quantstart.com/articles/Successful-Backtesting-of-Algorithmic-Trading-Strategies-Part-I

From Wikipedia:

"The only prior knowledge required is a list of variables which can be hypothesized to affect each other intertemporally."

http://en.wikipedia.org/wiki/Vector_autoregression

For example, one might think that the price change of General Motors today (GM) might depend on the recent prices of oil (USO is an oil ETF). This can be modeled using a tool such as this.

Note that there may not be a predictive relationship for your chosen symbols. In this case, a model's backtest will be poor (low % correct) and the model is not useful for forecasting.

In the case where a backtest yields reasonable results, the model may have some predictive power for 1 day ahead forecasts.

Only price travel Direction is attempted to be modeled, not the actual price change in dollars.

Quickstart:

1. enter symbol to forecast and candidate predictor symbols.

2. select Backtest or Forecast.

3. Press Run. Calculation results will appear in green window.

Further details at http://diffent.com/MktVecAR.pdf

Features:

Up to 6 candidate predictor stocks/ETFs (including target).

3 model types generated for every forecast for comparison purposes:

annealing classifier (slow or fast)
linear least squares
k-nearest classifier

1 and 2 day data lags automatically generated [AR(2) type models]

Up to 500 trading days of backtests.

1 day forward forecast.

Model can be built/forecasted before market opens since "today's" open prices are not included in the model.

Detailed log file of calculations.
Calculations done on a server for battery conservation.
Ability to Stop long calculations on the server.

Download the assembled regression tables with dates aligned in CSV format for further study.

Send downloaded tables to other apps on your device or via email.

Version history

7.2

2022-10-19

Fix crash on startup on older OS versions.

7.1

2022-10-14

Clean up tab bar icons.

7.0

2022-10-12

Add additional model settings to the output text for reference, the flags for normalize, daysWithheld, allowShorting, and the risk free rate.

6.9

2022-10-11

Change export button on light green output window (which is visible when you expand the window using the + button) to the standard share out icon. Add share out icon and feature to the plots screen to share out plots. Fix crash exporting output window on iPad.

6.6

2022-10-11

Add ability to withhold a number of days of data from the most recent end of the raw data set when building models. This is done by changing the number in the dayswthld field on the main screen of the app. For example, to hold back 5 trading days, set this value to 5. This lets you build models "in the past" instead of using all available data. Then when you forecast one day ahead, it forecasts one trading day into this withheld data block. You can use this feature to re-run similar models that you built on previous days with the exact same end point. And you can use this feature to check the forward one day forecasts separately from the built-in backtest features. E.g. you can forecast one day ahead and check the results on that day from another source such as Yahoo finance historical data. The app will tell you which particular day the "one day ahead forecast" refers to, adjusting for the new days withheld setting.

6.5

2022-10-05

Add ability to normalize X variables (on Tune screen). This is mostly useful when daily price changes in dollars of the assets specified (related to the nominal price level of the asset) are vastly different among the symbols you specify.

6.2

2022-09-28

Allow a new optimization method to be applied to model 1: BFGS. This is an oft-used method for nonlinear solvers; see https://en.wikipedia.org/wiki/Broyden–Fletcher–Goldfarb–Shanno_algorithm for more details. This is much faster than slow annealing in most cases, and may give better results in some cases.

6.1

2022-09-23

Report out the Sortino+ ratio for the paper trade backtest, which ratio is computed similarly to Sortino, but choosing only positive returns (instead of negative). This is annotated as "so+" in the model output window. This number is more directly comparable to the Sortino ratio than is the Sharpe ratio. For more information on this topic, see Section 6 of our technical whitepaper (sub section: A note on Sortino computations), accessible from the Help tab of the app.

6.0

2022-09-20

Report out Sharpe and Sortino ratios at the end of the backtest for the basic trading systems built on top of the 3 models.

Allow the user to enter the risk free rate to use for these ratios (this is located on the Tune screen of the app).

Since the model is based on daily data, all numbers are converted to daily units before computing the ratios, with approximations such as daily risk free rate = entered risk free rate divided by 252 (trading days per year). This is coded for the stock market; you can make appropriate adjustments for the 365 trading day/year crypto market until we generalize this.

Sortino ratio downside volatility is estimated using the method of zeroing out positive returns before computing the standard deviation of the (now all non-positive) return set (e.g. rather than deleting positive returns, we set them to zero per standard practice). This may not be ideal for comparing Sortino to Sharpe, but seems to be somewhat standard practice, so we present it as a start.
2,995
Keywords

5.9

2022-09-19

Report out the estimated percent returns over the backtest period of the the various models from the basic trading system recently added to the system. Report out some information on how this is computed such as the backtest start day, the opening price on that day, and the initial investment in dollars (based upon the aforementioned opening price, number of shares, and trade cost). This information is useful if one wishes to audit the computations. Also make some minor UI fixes to the first screen of the app.

Add plots of trading system outcomes over time versus buy and hold.

Make the output window bigger on taller phones.

5.6

2022-09-14

When doing a backtest, automatically apply a paper trading system to the forecasts made during the backtest period.

The user has 3 parameters to set for this:

A. share counts to trade (fractional allowed)
B. cost per trade in dollars (fractional dollars aka pennies allowed)
C. whether or not shorting shares should be allowed during trading

For this simple paper trade system, we do not take into account any possible extra costs of shorting shares
other than the per-trade cost. You may be able to to back-solve an adjusted trade cost
to account for any such additional shorting costs if you wish.

Theoretical results will improve if you use more shares since the percent of each transaction
consumed by the the cost per trade is reduced.

Paper trade system characteristics:

Buy and hold has two transactions (one buy at the beginning of
the backtest period, and then one sell at the end of the backtest period)
which subtracts two "cost per trade" amounts from the buy and hold results.

Buy and hold is listed as approx(imate) because it does not consider aftermarket trading
but only price movement during the trading day, and is mainly provided as a baseline
reference for the 3 models.

For the 3 models, on a day that a
trade is recommended, a position is opened
at the beginning of the trading day
(long) if the model signals a bullish day,
or (optionally) short if the model signals
a bearish day.

The position is closed out at the end of the
trading day to leave no open positions after hours.

This results in two trades per day on any day that a trade is recommended
(e.g. the cost per trade * 2 is subtracted from any gains or losses that day).

Cost per trade is currently assumed to be constant
no matter how many shares are traded.

This prototype paper trade system assumes that a trader could open a position at the beginning of the trading day at the opening price
and close it at the closing price at the end of the day. E.g. there is no modeling of illiquid markets or other technical glitches where
he could not trade at the modeled price.

Nonetheless, the system may be of some use in analyzing these type of forecasting models.

We currently only report out these prototype trade system results for the 3 final models
that are zero toleranced, since these usually yield
as good as or better statistical properties than the raw models
which forecast (trade) every day. If the app computes the
best zero tolerance to be 0.0 exactly for a given model, that model will
trade every day.

In this first implementation of a trial trading system backtest which uses
the bullish/bearish signals, we only report out the estimated dollar gain or loss
for simplicity (not percentage gain yet or Sharpe / Sortino ratio etc).

We also now allow you to export as a CSV file the forecasted results for all models during a backtest
in the Data & Info tab of the app.

Update screen shots to indicate new features.

5.5

2022-09-06

After a backtest is complete, report the bad predict sequence metrics (max and mean) using the zero-toleranced forecasts,
rather than the raw forecasts.

5.4

2022-09-03

On the backtest stairstep sequential fail plots, center the steps on the X axis integers. Now that there are dots on the plots indicating which days trades were recommended, it may be useful to know which stairstep applies to which dot.

5.3

2022-09-03

Add dots to backtest stairstep fail plots to indicate what days trades were recommended.

5.2

2022-09-02

Modify the bad run plots to show both the raw model results and the results filtered by the computed zero tolerance. Fix an error that was occurring when sending the zero tolerances back to the app. Zero tolerances sent back were double what was computed. This has been corrected.

5.0

2022-08-30

Add link to SSRN-hosted white paper for a deep dive into how the models in this app work, at top of Help screen. Sync up code base to match new macOS release of this app.

4.6

2022-08-27

Allow to run on iPad in full screen mode.

4.5

2022-08-26

Show some metrics on the light green output screen regarding the distribution of model term counts (feature counts) aggregated over the backtest and/or forecast.

Examples:

m1 term count 5,50,95 [5,7,10] of 11
m3 term count 5,50,95 [8,11,11] of 11

Examining the first line, this means that model 1's term count (out of a total of 11 possible terms) has a distribution throughout the backtest with a median or 50th percentile term count of 7. The 5th and 95th percentiles of the term count distribution are 5 and 10.

These metrics can give a hint if you are overfitting or underfitting your data. For example, if all of the numbers in the brackets show up as 1, it means that only 1 term (likely the constant in the linear class of models) has been chosen. This may hint to an underfit. In this case, it may be useful to reduce the windowsize on the Tune screen to reduce the complexity of the data that is trying to be fitted by a linear model. Conversely, if all numbers within the brackets are close to the max possible term count (in this case "of 11"), it may be that you are overfitting (this should also show up as a poor backtest). In that case, it may be worth trying to increase the point windowsize a bit.

4.4

2022-08-26

Show the tuning parameter settings from the Tune screen on the output window for easier reference if screen shots are taken.

4.3

2022-08-25

Provide better descriptions of front page app settings on the green output screen text for record keeping purposes in case you
forward the output to email or etc. This can help with model tuning efforts.

4.2

2022-08-22

Allow the % cutoff value on the Tune screen to work with model 1. This allows for a linear-in-coefficients model solved by optimization (either optimized to the original annealing solver target of max correct predictions in-sample, or the new minimum absolute error target [with the error to an optional fractional power as noted in the prior release]) to be term-reduced, where the least contributing terms are eliminated from the model; then the model is re-fit. This is similar to feature selection but in an abbreviated rather than exhaustive manner. To get full term models, set the % cutoff value to 0. Method: After a full term solve, the model is evaluated (not solved) at all in-sample points, which each variable set to 0 in turn. E.g. the model is evaluated with variable 1 = 0, variable 2 = 0, etc, and for each zeroed case, the optimization metric is stored. Terms that cause little change in the overall optimization metric when those terms are set to 0 are treated as low contributor terms and are removed from the model. The model is then re-fit with the reduced number of terms. The term reduction happens at each day of the backtest and/or forecast rather than universally over all backtests, e.g., model terms can blip into existence then disappear again as the backtest progresses.

4.1

2022-08-21

Add ability to modify the exponent on the residuals (errors) of the regression for the new max absolute deviation regression case. For example, setting this power to 2 yields ordinary least squares regression (though solved by an optimizer rather than matrix methods). Setting this power lower than 2 reduces the effect of outliers in the data (versus ordinary least squares regression). The default is 1.0 to mimic the plain least absolute deviation regression setup. This exponent allows some finer tuning of a linear-in-coefficients model. This field is marked with an x on the main panel of the app next to the Run button. Setting this power greater than 2 magnifies the effect of outliers in the model which may be useful in some cases. As always, the backtest results are the guides for tuning the models.

4.0

2022-08-19

Provide a least absolute difference regression option for model 1. This is
accomplished via a numeric optimizer approach. This may be contrasted to the
traditional least squared (difference) regression used for model 2 (e.g.
traditional linear regression). Model 2 is solved by traditional matrix methods
rather than numeric optimization.

Increase allowable solver iterations and allowable range of solved coefficients for
the Annealing (fast/slow) options for model 1. Reminder: The Anneal method merely
tries to solve for candidate predictor coefficients such that the sum of directionally correct
forecasts in-sample is maximized. However, the Annealing method does not necessarily
over-fit to that maximum in-sample fit. E.g. it does not necessary "cool" the max. Knowing
that overfitting is often a problem in machine learning, this is may be a benefit to
the model. For more information on the Anneal method, see: https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.anneal.html

3.9

2022-08-16

Add support for 11 more crypto currencies:

FLOW VET$ FIL ICP MANA  SAND$ XTZ HBAR AAVE THETA QNT

The final $ signs on some of these are to distinguish them from the similar stock symbol.

3.8

2022-08-14

Provide the option to set the app to a starter model that currently yields a reasonable 100 day backtest.