Skip to content

Commit e3c93d6

Browse files
peace-dovepeace-dove
committed
doc: recursive path filtering
1 parent 1881c58 commit e3c93d6

File tree

8 files changed

+941
-1
lines changed

8 files changed

+941
-1
lines changed

‎misc/experiments.md‎

Lines changed: 101 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,5 +28,105 @@ Docker image: tugraph/tugraph-compile-centos8:1.3.1
2828
- Baseline:
2929
- Optimized:
3030
- Datasets:
31+
- SF100：https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/finbench/v0.2.0/sf100/sf100.tar.xz
32+
- Parameter of SF100：https://tugraph-web.oss-cn-beijing.aliyuncs.com/tugraph/datasets/finbench/v0.2.0/sf100/substitute_parameters.tar.gz.
3133

32-
## Experiment Steps
34+
## Experiment Steps
35+
36+
### Move to directory
37+
Download dataset and parameter and move them to the dataset directory, for example, the directory '/data/dataset'.
38+
39+
Modify docker-compose.yml to attach local file to volume '/root/datasets/' in container, for example,
40+
```
41+
volumes:
42+
- /data/dataset:/root/datasets/
43+
- ./scripts:/root/scripts/
44+
```
45+
46+
Start docker containers.
47+
```
48+
docker-compose up -d
49+
```
50+
51+
After that, will see two containers.
52+
- [directory name]_tugraph_1
53+
- [directory name]_finbench_1
54+
55+
Benchmark is based on a large-scale SF100 dataset, and the following scripts are required.
56+
57+
### Install package
58+
First, attach to the 'finbench-easyrun_tugraph_1' container.
59+
60+
Then install the package.
61+
```
62+
rpm -i base.rpm
63+
```
64+
65+
### Import data
66+
Enter the '/root/scripts' directory.
67+
```
68+
bash sf100_import.sh
69+
```
70+
71+
### Start tugraph server
72+
Also in the same container 'finbench-easyrun_tugraph_1' and same directory'/root/scripts', start the server.
73+
```
74+
bash sf100_start.sh
75+
```
76+
77+
### C++ Plugin
78+
Switch to another container 'finbench'.
79+
80+
Enter the '/root/scripts' directory.
81+
```
82+
bash load_procedure.sh
83+
```
84+
To install C++ plugins.
85+
86+
### Benchmark Baseline
87+
Before benchmark, in the container 'finbench', copy the file 'benchmark_sf100.properties' in scripts to the directory '/root/ldbc/ldbc_finbench_transaction_impls/tugraph/benchmark.properties'.
88+
89+
Also in the 'finbench' container and enter the '/root/scripts' directory.
90+
```
91+
bash sf100_benchmark.sh
92+
```
93+
94+
### Benchmark Opt
95+
Attach to container 'tugraph'.
96+
97+
Uninstall the baseline package.
98+
```
99+
rpm -e tugraph-4.0.0-1.x86_64
100+
```
101+
Install the opt version.
102+
```
103+
rpm -i opt.rpm
104+
```
105+
106+
Switch to '/root/scripts'.
107+
108+
Delete the old data directory.
109+
110+
```
111+
rm -r /root/lgraph_db_sf100
112+
rm -r /root/lgraph_log_sf100
113+
114+
bash sf100_import.sh
115+
bash sf100_start.sh
116+
```
117+
118+
Attach to 'finbench' container, and enter '/root/scripts'.
119+
```
120+
bash load_procedure.sh
121+
```
122+
123+
Before benchmark, comment the line
124+
```
125+
# cd data && ln -s /root/datasets/sf100 ./sf100 && cd ..
126+
```
127+
in the file 'sf100_benchmark.properties'
128+
129+
Then run the benchmark.
130+
```
131+
bash sf100_benchmark.sh
132+
```

‎misc/opt_recursive/docker-compose.yml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
version: "3"
2+
services:
3+
tugraph:
4+
image: tugraph/tugraph-compile-centos8:1.2.9
5+
command: tail -f
6+
tty: true
7+
volumes:
8+
- /data/dataset:/root/datasets/
9+
- ./scripts:/root/scripts/
10+
expose:
11+
- 7070
12+
- 9090
13+
14+
finbench:
15+
depends_on:
16+
- tugraph
17+
image: tugraph/tugraph-bdci-finbench:cent8
18+
command: tail -f
19+
tty: true
20+
volumes:
21+
- /data/dataset:/root/datasets/
22+
- ./scripts:/root/scripts/

‎misc/opt_recursive/scripts/benchmark_sf100.properties‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
############################################################
2+
# SUT defined configurations #
3+
############################################################
4+
uri=list://172.21.189.228:9090
5+
user=admin
6+
pass=73@TuGraph
7+
8+
############################################################
9+
# Driver configurations #
10+
############################################################
11+
status=1
12+
thread_count=16
13+
name=LDBC-FinBench
14+
# Modes available: 1.CREATE_VALIDATION 2.VALIDATE_DATABASE 3.EXECUTE_BENCHMARK
15+
mode=EXECUTE_BENCHMARK
16+
results_log=true
17+
time_unit=MICROSECONDS
18+
time_compression_ratio=0.1
19+
peer_identifiers=
20+
workload_statistics=false
21+
spinner_wait_duration=1
22+
help=false
23+
ignore_scheduled_start_times=false
24+
workload=org.ldbcouncil.finbench.driver.workloads.transaction.LdbcFinBenchTransactionWorkload
25+
db=org.ldbcouncil.finbench.impls.tugraph.TuGraphTransactionDb
26+
operation_count=1800000
27+
validation_parameters_size=100
28+
validate_workload=true
29+
validate_database=validation_params.csv
30+
warmup=450000
31+
ldbc.finbench.transaction.queries.parameters_dir=data/sf100/read_params
32+
ldbc.finbench.transaction.queries.updates_dir=data/sf100/incremental
33+
# param and update files suffix, `csv` or `parquet`, default is `csv`
34+
ldbc.finbench.transaction.queries.files_suffix=csv
35+
ldbc.finbench.transaction.queries.simple_read_dissipation=0.2
36+
ldbc.finbench.transaction.queries.update_interleave=500
37+
ldbc.finbench.transaction.queries.scale_factor=1
38+
# Frequency of complex read queries
39+
ldbc.finbench.transaction.queries.ComplexRead1_freq=12
40+
ldbc.finbench.transaction.queries.ComplexRead2_freq=45
41+
ldbc.finbench.transaction.queries.ComplexRead3_freq=23
42+
ldbc.finbench.transaction.queries.ComplexRead4_freq=57
43+
ldbc.finbench.transaction.queries.ComplexRead5_freq=83
44+
ldbc.finbench.transaction.queries.ComplexRead6_freq=21
45+
ldbc.finbench.transaction.queries.ComplexRead7_freq=36
46+
ldbc.finbench.transaction.queries.ComplexRead8_freq=51
47+
ldbc.finbench.transaction.queries.ComplexRead9_freq=21
48+
ldbc.finbench.transaction.queries.ComplexRead10_freq=6
49+
ldbc.finbench.transaction.queries.ComplexRead11_freq=12
50+
ldbc.finbench.transaction.queries.ComplexRead12_freq=34
51+
# For debugging purposes
52+
ldbc.finbench.transaction.queries.ComplexRead1_enable=true
53+
ldbc.finbench.transaction.queries.ComplexRead2_enable=true
54+
ldbc.finbench.transaction.queries.ComplexRead3_enable=true
55+
ldbc.finbench.transaction.queries.ComplexRead4_enable=true
56+
ldbc.finbench.transaction.queries.ComplexRead5_enable=true
57+
ldbc.finbench.transaction.queries.ComplexRead6_enable=true
58+
ldbc.finbench.transaction.queries.ComplexRead7_enable=true
59+
ldbc.finbench.transaction.queries.ComplexRead8_enable=true
60+
ldbc.finbench.transaction.queries.ComplexRead9_enable=true
61+
ldbc.finbench.transaction.queries.ComplexRead10_enable=true
62+
ldbc.finbench.transaction.queries.ComplexRead11_enable=true
63+
ldbc.finbench.transaction.queries.ComplexRead12_enable=true
64+
ldbc.finbench.transaction.queries.SimpleRead1_enable=true
65+
ldbc.finbench.transaction.queries.SimpleRead2_enable=true
66+
ldbc.finbench.transaction.queries.SimpleRead3_enable=true
67+
ldbc.finbench.transaction.queries.SimpleRead4_enable=true
68+
ldbc.finbench.transaction.queries.SimpleRead5_enable=true
69+
ldbc.finbench.transaction.queries.SimpleRead6_enable=true
70+
ldbc.finbench.transaction.queries.Write1_enable=true
71+
ldbc.finbench.transaction.queries.Write2_enable=true
72+
ldbc.finbench.transaction.queries.Write3_enable=true
73+
ldbc.finbench.transaction.queries.Write4_enable=true
74+
ldbc.finbench.transaction.queries.Write5_enable=true
75+
ldbc.finbench.transaction.queries.Write6_enable=true
76+
ldbc.finbench.transaction.queries.Write7_enable=true
77+
ldbc.finbench.transaction.queries.Write8_enable=true
78+
ldbc.finbench.transaction.queries.Write9_enable=true
79+
ldbc.finbench.transaction.queries.Write10_enable=true
80+
ldbc.finbench.transaction.queries.Write11_enable=true
81+
ldbc.finbench.transaction.queries.Write12_enable=true
82+
ldbc.finbench.transaction.queries.Write13_enable=true
83+
ldbc.finbench.transaction.queries.Write14_enable=true
84+
ldbc.finbench.transaction.queries.Write15_enable=true
85+
ldbc.finbench.transaction.queries.Write16_enable=true
86+
ldbc.finbench.transaction.queries.Write17_enable=true
87+
ldbc.finbench.transaction.queries.Write18_enable=true
88+
ldbc.finbench.transaction.queries.Write19_enable=true
89+
ldbc.finbench.transaction.queries.ReadWrite1_enable=true
90+
ldbc.finbench.transaction.queries.ReadWrite2_enable=true
91+
ldbc.finbench.transaction.queries.ReadWrite3_enable=true

0 commit comments

Comments
 (0)