HFDataViewSignalResearch
一、 数据读取与处理
二、 画图观察与构思算法
三、 进场&时间出场&&止损止盈
- 数据读取并转换成jaqs需要的数据格式
- 数据合并成不同周期
1. reset_index
2. datetime to int
3. add symbol column
import pandas as pd
HFData = pd.read_excel('HFData1M.xlsx').set_index('datetime')# reset_index
# dataResample
def dt2int(t):
return t.year*10**10+t.month*10**8+t.day*10**6+t.hour*10**4+t.minute*10**2+t.second
def modifyDf(data, symbol):
assert isinstance(data, pd.DataFrame)
data = data.reset_index()
data['trade_date'] = data['datetime'].apply(dt2int)
data['symbol'] = symbol
return datadataIf = modifyDf(HFData, 'IF88_CTP')dataIf.tail()| datetime | close | date | exchange | high | low | open | openInterest | symbol | time | trading_date | volume | vtSymbol | trade_date | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 43435 | 2019-02-28 14:55:00 | 3678.8 | 20190228 | CTP | 3680.4 | 3678.6 | 3679.6 | 69005 | IF88_CTP | 14:55:00 | 2019-02-28 | 292 | IF88:CTP | 20190228145500 |
| 43436 | 2019-02-28 14:56:00 | 3680.0 | 20190228 | CTP | 3681.4 | 3678.6 | 3678.8 | 69149 | IF88_CTP | 14:56:00 | 2019-02-28 | 344 | IF88:CTP | 20190228145600 |
| 43437 | 2019-02-28 14:57:00 | 3682.0 | 20190228 | CTP | 3683.0 | 3679.6 | 3679.8 | 69314 | IF88_CTP | 14:57:00 | 2019-02-28 | 437 | IF88:CTP | 20190228145700 |
| 43438 | 2019-02-28 14:58:00 | 3679.6 | 20190228 | CTP | 3684.4 | 3679.6 | 3682.0 | 69540 | IF88_CTP | 14:58:00 | 2019-02-28 | 518 | IF88:CTP | 20190228145800 |
| 43439 | 2019-02-28 14:59:00 | 3677.4 | 20190228 | CTP | 3681.2 | 3677.2 | 3679.6 | 69794 | IF88_CTP | 14:59:00 | 2019-02-28 | 605 | IF88:CTP | 20190228145900 |
1. resample
2. agg
def resampleDf(data, freq='H'):
return data.resample(freq).agg({'open':'first', 'high': 'max', 'low': 'min', 'close': 'last', 'volume':'sum'})
dataResample = resampleDf(HFData, '15Min')
dataIf15 = modifyDf(dataResample, 'IF88_CTP')dataIf15.tail()| datetime | open | high | low | close | volume | trade_date | symbol | |
|---|---|---|---|---|---|---|---|---|
| 26129 | 2019-02-28 13:45:00 | 3676.4 | 3681.6 | 3668.0 | 3668.6 | 4641.0 | 20190228134500 | IF88_CTP |
| 26130 | 2019-02-28 14:00:00 | 3668.6 | 3683.8 | 3667.2 | 3679.2 | 3424.0 | 20190228140000 | IF88_CTP |
| 26131 | 2019-02-28 14:15:00 | 3679.2 | 3680.8 | 3664.2 | 3678.6 | 4146.0 | 20190228141500 | IF88_CTP |
| 26132 | 2019-02-28 14:30:00 | 3679.0 | 3687.4 | 3668.4 | 3671.2 | 4397.0 | 20190228143000 | IF88_CTP |
| 26133 | 2019-02-28 14:45:00 | 3671.2 | 3684.4 | 3667.2 | 3677.4 | 4624.0 | 20190228144500 | IF88_CTP |
# 整数转Datetime
from datetime import datetime
dataIf15['datetimeType'] = list(map(lambda x : datetime.strptime(str(x), "%Y%m%d%H%M%S"), dataIf15.trade_date))- 安装jaqs_fxdayu
- 将数据导入HFDataView
- 算法使用与画图观察
- talib
- 画图去除停牌时间
- 观察并设计算法
pip install git+https://github.com/xingetouzi/jaqs-fxdayu.git
# import
from jaqs_fxdayu.data.hf_dataview import HFDataViewD:\Anaconda3\lib\importlib\__init__.py:126: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
return _bootstrap._gcd_import(name[level:], package, level)
1. instantiation
2. create_inti_dv
# instantiation
dv = HFDataView()
# create_inti_dv
dv.create_init_dv(dataIf.dropna().set_index(['trade_date', 'symbol']))Initialize dataview success.
dv15 = HFDataView()
dv15.create_init_dv(dataIf15.dropna().set_index(['trade_date', 'symbol']))Initialize dataview success.
# 查看函数
dv.func_doc.funcsarray(['+', '-', '*', '/', 'Sign(x)', 'Abs(x)', 'Log(x)', '-x', '^',
'Pow(x,y)', 'SignedPower(x,e)', '%', '==', '!=', '>', '<', '>=',
'<=', '&&', '||', '!', 'IsNan(x)', 'Sin(x)', 'Cos(x)', 'Tan(x)',
'Sqrt(x)', 'Ceil(x)', 'Floor(x)', 'Round(x)', 'Max(x,y)',
'Min(x,y)', 'If(cond,x,y)', 'Delay(x,n)', 'Ts_Sum(x,n)',
'Ts_Product(x,n)', 'Delta(x,n)', 'Return(x,n,log)', 'Ts_Mean(x，n)',
'StdDev(x,n)', 'Covariance(x,y,n)', 'Correlation(x,y,n)',
'Ts_Min(x，n)', 'Ts_Max(x，n)', 'Ts_Skewness(x，n)',
'Ts_Kurtosis(x，n)', 'Ts_Rank(x, n)', 'Ts_Percentile(x, n)',
'Ts_Quantile(x, n)', 'Ewma(x, halflife)', 'Rank(x)',
'GroupRank(x,g)', 'Percentile(x)', 'GroupPercentile(x, g, n)',
'ConditionRank(x, cond)', 'Quantile(x, n)',
'GroupQuantile(x, g, n)', 'Standardize(x)', 'Cutoff(x, z_score)',
'CumToSingle(x)', 'TTM(x)', 'Decay_exp(x,f,n)',
'Decay_linear(x,n)', 'Tail(x, lower, upper, newval)', 'Step(n)',
'CountNans(x,n)', 'Ts_Argmax(x,n)', 'Ts_Argmin(x,n)',
'Ta(ta_method,ta_column,open,high,low,close,volume,*args)'],
dtype=object)
# 查看函数定义
dv.func_doc.doc[dv.func_doc.doc['公式']=='If(cond,x,y)']| 分类 | 说明 | 公式 | 示例 | |
|---|---|---|---|---|
| 31 | 选择函数 | cond为True取x的值，反之取y的值 | If(cond,x,y) | If(close > open, close, open) 表示取open和close的较大值 |
dv.data.tail()| symbol | IF88_CTP | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| field | close | date | datetime | exchange | high | low | open | openInterest | time | trading_date | volume | vtSymbol |
| trade_date | ||||||||||||
| 20190228145500 | 3678.8 | 20190228 | 2019-02-28 14:55:00 | CTP | 3680.4 | 3678.6 | 3679.6 | 69005 | 14:55:00 | 2019-02-28 | 292 | IF88:CTP |
| 20190228145600 | 3680.0 | 20190228 | 2019-02-28 14:56:00 | CTP | 3681.4 | 3678.6 | 3678.8 | 69149 | 14:56:00 | 2019-02-28 | 344 | IF88:CTP |
| 20190228145700 | 3682.0 | 20190228 | 2019-02-28 14:57:00 | CTP | 3683.0 | 3679.6 | 3679.8 | 69314 | 14:57:00 | 2019-02-28 | 437 | IF88:CTP |
| 20190228145800 | 3679.6 | 20190228 | 2019-02-28 14:58:00 | CTP | 3684.4 | 3679.6 | 3682.0 | 69540 | 14:58:00 | 2019-02-28 | 518 | IF88:CTP |
| 20190228145900 | 3677.4 | 20190228 | 2019-02-28 14:59:00 | CTP | 3681.2 | 3677.2 | 3679.6 | 69794 | 14:59:00 | 2019-02-28 | 605 | IF88:CTP |
1. add_formula
2. get_ts
# add_formula
gap = dv.add_formula('Gap','close-Delay(open, 1)', add_data=True)# dv.add_formula?# algorithm
gap.tail()| symbol | IF88_CTP |
|---|---|
| trade_date | |
| 20190228145500 | -1.6 |
| 20190228145600 | 0.4 |
| 20190228145700 | 3.2 |
| 20190228145800 | -0.2 |
| 20190228145900 | -4.6 |
# get_ts
gapDt = dv.get_ts('Gap', date_type='datetime')# dv.get_ts?gapDt.tail()| symbol | IF88_CTP |
|---|---|
| trade_date | |
| 2019-02-28 14:55:00 | -1.6 |
| 2019-02-28 14:56:00 | 0.4 |
| 2019-02-28 14:57:00 | 3.2 |
| 2019-02-28 14:58:00 | -0.2 |
| 2019-02-28 14:59:00 | -4.6 |
import matplotlib.pyplot as plt
fig = plt.figure(figsize=(15,7))
plt.plot(gapDt)
plt.show()
dv.add_formula('name', "Ta('Function',0,open, high, low, close, volume)",add_data=True)
macdDv = dv15.add_formula('macd', "Ta('MACD',0,open, high, low, close, volume, 12, 26, 9)",add_data=True)
macdSignalDv = dv15.add_formula('macdSignal', "Ta('MACD',1,open, high, low, close, volume, 12, 26, 9)",add_data=True)
macdHistDv = dv15.add_formula('macdHist', "Ta('MACD',2,open, high, low, close, volume, 12, 26, 9)",add_data=True)# 生成DataFrame
DataDf = pd.DataFrame(
{
'close':dv15.get_ts('close', date_type = 'datetime')['IF88_CTP'],
'macd' : dv15.get_ts('macd', date_type='datetime')['IF88_CTP'],
'macdSignal':dv15.get_ts('macdSignal', date_type='datetime')['IF88_CTP'],
'macdHist':dv15.get_ts('macdHist', date_type='datetime')['IF88_CTP']
}
)def chartRange(df, n=100):
fig, (ax, ax1) = plt.subplots(2, 1, sharex=True, figsize=(15,10))
ax.plot(df.index[-n:], DataDf.close.iloc[-n:])
ax1.plot(df.index[-n:], DataDf.macd.iloc[-n:])
ax1.plot(df.index[-n:], DataDf.macdSignal.iloc[-n:])
ax1.bar(df.index[-n:], DataDf.macdHist.iloc[-n:], color='green', width=0.8)
plt.show()chartRange(DataDf, 5000)
1. reset_index
2. strftime时间格式转成字符串
3. xticks设置x轴索引
dataIf1 = DataDf.reset_index()# strftime时间格式转成字符串
dataIf1['tradeDateStr'] = list(map(lambda x: x.strftime(format='%Y-%m-%d %H:%M:%S'), list(dataIf1.trade_date)))dataIf1.tail()| trade_date | close | macd | macdHist | macdSignal | tradeDateStr | |
|---|---|---|---|---|---|---|
| 2891 | 2019-02-28 13:45:00 | 3668.6 | -5.885688 | -2.157227 | -3.728461 | 2019-02-28 13:45:00 |
| 2892 | 2019-02-28 14:00:00 | 3679.2 | -6.112784 | -1.907458 | -4.205326 | 2019-02-28 14:00:00 |
| 2893 | 2019-02-28 14:15:00 | 3678.6 | -6.268910 | -1.650868 | -4.618043 | 2019-02-28 14:15:00 |
| 2894 | 2019-02-28 14:30:00 | 3671.2 | -6.910104 | -1.833649 | -5.076455 | 2019-02-28 14:30:00 |
| 2895 | 2019-02-28 14:45:00 | 3677.4 | -6.839130 | -1.410140 | -5.428990 | 2019-02-28 14:45:00 |
def chartRange(data, n=100):
fig, (ax, ax1) = plt.subplots(2, 1, sharex=True, figsize=(15,10))
ax.plot(data.index[-n:], data.close.iloc[-n:])
ax1.plot(data.index[-n:], data.macd.iloc[-n:])
ax1.plot(data.index[-n:], data.macdSignal.iloc[-n:])
ax1.bar(data.index[-n:], data.macdHist.iloc[-n:], color='green', width=0.8)
plt.xticks([data.index[-n:][0], data.index[-n:][int(n/2)], data.index[-n:][-1]],
[data.tradeDateStr.iloc[0],data.tradeDateStr.iloc[int(n/2)], data.tradeDateStr.iloc[-1]])
plt.show()chartRange(dataIf1, n=1000)
1. 0轴区分
2. 金叉死叉
3. 二次金叉与死叉
4. macd与macdSignal同涨同跌
5. macdHist的下降与上升
6. macdHist第五元素
策略一 long: (macd[-1]>0 and goldenCross)
short: (macd[-1]<0 and deathCross)
策略二 long: macd[-1]>0 and macd[-2]<0 and macd[-1]>macd[-3] and macdSignal[-1]>macdSignal[-3]
short: macd[-1]<0 and macd[-2]>0 and macd[-1]<macd[-3] and macdSignal[-1]<macdSignal[-3]
策略三 long: macd[-1]<0 and secondGoldenCross short: macd[-1]>0 and secondDeathCross
策略四 long: macd[-1]<0 and macdHistFiveUp
short: macd[-1]>0 and macdHistFiveDn
策略五 long: macd[-1]>0 and macdHistFiveDn and macdHistUp
short: macd[-1]<0 and macdHistFiveUp and macdHistDn
1. 将策略一在15Min周期下构建信号
2. 设置固定持有时间/设置止损止盈
3. 信号绩效评估
# 进场信号
long = dv15.add_formula("long","If(macd>0 && (Delay(macd,2)<=Delay(macdSignal,2) && macd>macdSignal),2,0)", add_data=True)
short = dv15.add_formula("short","If(macd<0 && (Delay(macd,2)>=Delay(macdSignal,2) && macd<macdSignal),-2,0)", add_data=True)
# 出场信号
close_long = dv15.add_formula("closeLong","If(short==-2,1,0)", add_data=True)
close_short = dv15.add_formula("closeShort","If(long==2,-1,0)", add_data=True)D:\Anaconda3\lib\site-packages\jaqs\data\py_expression_eval.py:477: RuntimeWarning: invalid value encountered in greater
res = arr > brr
D:\Anaconda3\lib\site-packages\jaqs\data\py_expression_eval.py:504: RuntimeWarning: invalid value encountered in less_equal
res = arr <= brr
D:\Anaconda3\lib\site-packages\jaqs\data\py_expression_eval.py:486: RuntimeWarning: invalid value encountered in less
res = arr < brr
D:\Anaconda3\lib\site-packages\jaqs\data\py_expression_eval.py:495: RuntimeWarning: invalid value encountered in greater_equal
res = arr >= brr
# long.iloc[800:-500]from jaqs_fxdayu.research import TimingDigger
# step 1：实例化TimingDigger 通过output_folder和output_format指定测试报告的输出路径和输出格式，通过signal_name指定绩效文件名称
td1 = TimingDigger(output_folder=".", output_format='pdf',signal_name='macdSignal1')
tdSlTp = TimingDigger(output_folder=".", output_format='pdf',signal_name='macdSignal1SlTp')
def TimingSignal(instantiation=td1, mhp=None, sl=None, tp=None):
#多空分别计算一遍 输出汇总结果
instantiation.process_signal(
enter_signal=dv15.get_ts("long"),
exit_signal=dv15.get_ts("closeLong"),
sig_type="long", # 信号类型 long/short
price=dv15.get_ts("close"),
max_holding_period=mhp, # 最大持有天数 可为空
stoploss=-sl if sl else None, # 止损百分比 负数 可为空
stopprofit=None, # 止盈百分比 正数 可为空
)
instantiation.process_signal(
enter_signal=dv15.get_ts("short"),
exit_signal=dv15.get_ts("closeShort"),
sig_type="short", # 信号类型 long/short
price=dv15.get_ts("close"),
max_holding_period=mhp, # 最大持有Bar数 可为空
stoploss=-sl if sl else None, # 止损百分比 负数 可为空
stopprofit=None, # 止盈百分比 正数 可为空
)D:\Anaconda3\lib\importlib\__init__.py:126: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
return _bootstrap._gcd_import(name[level:], package, level)
D:\Anaconda3\lib\site-packages\matplotlib\__init__.py:1405: UserWarning:
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.
warnings.warn(_use_error_msg)
# td1.process_signal?TimingSignal(td1, 50)Nan Data Count (should be zero) : 0; Percentage of effective data: 4%
Nan Data Count (should be zero) : 0; Percentage of effective data: 4%
td1.create_event_report(sig_type="long",
by_symbol=True)Figure saved: C:\Users\small\Desktop\June2019\时间序列课件\HFDataviewSignalDigger\macdSignal1/long_entry_exit_position/IF88_CTP.pdf
*****-Summary-*****
Event Analysis
win loss all
t-stat 8.458 -10.150 0.962
p-value 0.000 0.000 0.338
mean 0.020 -0.012 0.002
std 0.017 0.010 0.021
info_ratio 1.151 -1.222 0.086
skewness 0.728 -1.183 0.706
kurtosis -0.879 0.540 0.323
pct5 0.002 -0.036 -0.032
pct25 0.005 -0.017 -0.011
pct50 0.014 -0.010 -0.002
pct75 0.034 -0.005 0.012
pct95 0.051 -0.002 0.048
occurance 55.000 70.000 125.000
win_ratio NaN NaN 0.440
win_mean/loss_mean NaN NaN 1.608
td1.create_event_report(sig_type="short",
by_symbol=True)Figure saved: C:\Users\small\Desktop\June2019\时间序列课件\HFDataviewSignalDigger\macdSignal1/short_entry_exit_position/IF88_CTP.pdf
*****-Summary-*****
Event Analysis
win loss all
t-stat 8.879 -13.747 1.112
p-value 0.000 0.000 0.268
mean 0.025 -0.009 0.002
std 0.017 0.006 0.019
info_ratio 1.480 -1.598 0.106
skewness 0.723 -0.344 1.348
kurtosis -0.298 -1.177 1.227
pct5 0.002 -0.019 -0.018
pct25 0.017 -0.015 -0.010
pct50 0.023 -0.008 -0.004
pct75 0.026 -0.004 0.016
pct95 0.057 -0.002 0.049
occurance 37.000 75.000 112.000
win_ratio NaN NaN 0.330
win_mean/loss_mean NaN NaN 2.696
td1.create_event_report(sig_type="long_short",
by_symbol=True)Figure saved: C:\Users\small\Desktop\June2019\时间序列课件\HFDataviewSignalDigger\macdSignal1/long_short_entry_exit_position/IF88_CTP.pdf
*****-Summary-*****
Event Analysis
win loss all
t-stat 12.099 -15.445 1.457
p-value 0.000 0.000 0.146
mean 0.022 -0.011 0.002
std 0.017 0.008 0.020
info_ratio 1.268 -1.287 0.095
skewness 0.693 -1.390 0.964
kurtosis -0.665 2.011 0.701
pct5 0.002 -0.029 -0.020
pct25 0.006 -0.015 -0.011
pct50 0.018 -0.009 -0.003
pct75 0.033 -0.005 0.013
pct95 0.054 -0.002 0.049
occurance 92.000 145.000 237.000
win_ratio NaN NaN 0.388
win_mean/loss_mean NaN NaN 2.037
TimingSignal(tdSlTp, 50, 0.02, 0.08)Nan Data Count (should be zero) : 0; Percentage of effective data: 4%
Nan Data Count (should be zero) : 0; Percentage of effective data: 4%
tdSlTp.create_event_report(sig_type="long_short",
by_symbol=True)Figure saved: C:\Users\small\Desktop\June2019\时间序列课件\HFDataviewSignalDigger\macdSignal1SlTp/long_short_entry_exit_position/IF88_CTP.pdf
*****-Summary-*****
Event Analysis
win loss all
t-stat 12.383 -18.265 1.468
p-value 0.000 0.000 0.144
mean 0.023 -0.011 0.002
std 0.017 0.007 0.020
info_ratio 1.320 -1.506 0.096
skewness 0.672 -0.452 1.131
kurtosis -0.697 -1.058 0.671
pct5 0.002 -0.022 -0.022
pct25 0.008 -0.016 -0.011
pct50 0.019 -0.009 -0.004
pct75 0.033 -0.005 0.013
pct95 0.054 -0.002 0.049
occurance 89.000 148.000 237.000
win_ratio NaN NaN 0.376
win_mean/loss_mean NaN NaN 2.145