Getting Started
Getting started data aggregation and streaming with daggy
Getting Daggy
Windows
MacOS
1
brew install https://raw.githubusercontent.com/synacker/daggy_homebrew/master/daggy.rb
Copied!
Fedora
1
sudo dnf install daggy daggy-devel
Copied!
Linux
All daggy dependencies (Qt5, libssh2, libyaml-cpp) are included into separate dir and does not effect system applications.
Build from source
Environment requirenments
Build steps
1
git clone https://github.com/synacker/daggy.git
2
mkdir build
3
cd build
4
conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan
5
conan install ../daggy --build=missing
6
conan build ../daggy
7
cmake install
Copied!
Check installation
1
daggy --help
2
Usage: daggy [options] *.yaml|*.yml|*.json
3
​
4
Options:
5
-o, --output <folder> Set output folder
6
-f, --format <json|yaml> Source format
7
-i, --stdin Read data aggregation sources from stdin
8
-t, --timeout <time in ms> Auto complete timeout
9
-h, --help Displays help on commandline options.
10
--help-all Displays help including Qt specific options.
11
-v, --version Displays version information.
12
​
13
Arguments:
14
file data aggregation sources file
Copied!
Getting Started with data aggregation and streaming
Simple Data Aggregation Source
Create yaml or json file
YAML
JSON
simple.yaml
1
sources:
2
localhost:
3
type: local
4
commands:
5
pingYa:
6
exec: ping ya.ru
7
extension: log
Copied!
simple.json
1
{
2
"sources": {
3
"localhost": {
4
"commands": {
5
"pingYa": {
6
"extension": "log",
7
"exec": "ping ya.ru"
8
}
9
},
10
"type": "local"
11
}
12
}
13
}
Copied!
Run daggy
YAML
JSON
1
daggy simple.yaml
Copied!
1
daggy simple.json
Copied!
Check console output
1
23:07:23:977 | AppStat | Start aggregation in 01-04-20_23-07-23-977_simple
2
23:07:23:977 | ProvStat | localhost | New state: Started
3
23:07:23:977 | CommStat | localhost | pingYa | New state: Starting
4
23:07:23:977 | CommStat | localhost | pingYa | New state: Started
Copied!
There are all commands from simple.yaml/simple.json are streams in 01-04-20_23-07-23-977_simple with output files
Tailing streams from Simple Data Source
1
tail -f 01-04-20_23-07-23-977_simple/*
2
64 bytes from ya.ru (87.250.250.242): icmp_seq=99 ttl=249 time=21.2 ms
3
64 bytes from ya.ru (87.250.250.242): icmp_seq=100 ttl=249 time=18.8 ms
4
64 bytes from ya.ru (87.250.250.242): icmp_seq=101 ttl=249 time=23.5 ms
5
64 bytes from ya.ru (87.250.250.242): icmp_seq=102 ttl=249 time=18.8 ms
6
64 bytes from ya.ru (87.250.250.242): icmp_seq=103 ttl=249 time=18.8 ms
7
64 bytes from ya.ru (87.250.250.242): icmp_seq=104 ttl=249 time=17.4 ms
8
64 bytes from ya.ru (87.250.250.242): icmp_seq=105 ttl=249 time=17.4 ms
9
64 bytes from ya.ru (87.250.250.242): icmp_seq=106 ttl=249 time=20.1 ms
10
64 bytes from ya.ru (87.250.250.242): icmp_seq=107 ttl=249 time=25.8 ms
11
64 bytes from ya.ru (87.250.250.242): icmp_seq=108 ttl=249 time=35.1 ms
12
64 bytes from ya.ru (87.250.250.242): icmp_seq=109 ttl=249 time=21.1 ms
Copied!
Stop data aggregation and streaming
Type CTRL+C for stopping data aggregation and streaming. Type CTRL+C twice for hard stop application, without waiting cancelation of child local and remote processes.
1
23:07:23:977 | AppStat | Start aggregation in 01-04-20_23-07-23-977_simple
2
23:07:23:977 | ProvStat | localhost | New state: Started
3
23:07:23:977 | CommStat | localhost | pingYa | New state: Starting
4
23:07:23:977 | CommStat | localhost | pingYa | New state: Started
5
^C23:17:56:667 | ProvStat | localhost | New state: Finishing
6
23:17:56:668 | CommStat | localhost | pingYa | New state: Finished. Exit code: 0
7
23:17:56:668 | ProvStat | localhost | New state: Finished
8
23:17:56:668 | AppStat | Stop aggregation in 01-04-20_23-07-23-977_simple
Copied!
Investigate aggregated data
1
ls -l 01-04-20_23-07-23-977_simple/
2
-rw-r--r-- 1 muxa muxa 45574 апр 1 23:17 localhost-pingYa.log
Copied!
Example of Data Aggregation Sources with multiple commands and remote data aggregation and streaming
YAML
JSON
multiple_and_remote.yaml
1
aliases:
2
- &my_commands
3
pingYa:
4
exec: ping ya.ru
5
extension: log
6
pingGoo:
7
exec: ping goo.gl
8
extension: log
9
​
10
- &ssh_auth
11
user: muxa
12
key: /home/muxa/.ssh/id_rsa
13
​
14
sources:
15
localhost:
16
type: local
17
commands: *my_commands
18
remotehost:
19
host: 192.168.1.9
20
type: ssh2
21
parameters: *ssh_auth
22
commands: *my_commands
23
remotehost2:
24
host: 192.168.1.9
25
type: ssh2
26
parameters: *ssh_auth
27
commands: *my_commands
28
remotehost3:
29
host: 192.168.1.9
30
type: ssh2
31
parameters: *ssh_auth
32
commands: *my_commands
Copied!
multiple_and_remote.json
1
{
2
"sources": {
3
"remotehost3": {
4
"parameters": {
5
"user": "muxa",
6
"key": "/home/muxa/.ssh/id_rsa"
7
},
8
"commands": {
9
"pingYa": {
10
"extension": "log",
11
"exec": "ping ya.ru"
12
},
13
"pingGoo": {
14
"extension": "log",
15
"exec": "ping goo.gl"
16
}
17
},
18
"type": "ssh2",
19
"host": "192.168.1.9"
20
},
21
"remotehost2": {
22
"parameters": {
23
"user": "muxa",
24
"key": "/home/muxa/.ssh/id_rsa"
25
},
26
"commands": {
27
"pingYa": {
28
"extension": "log",
29
"exec": "ping ya.ru"
30
},
31
"pingGoo": {
32
"extension": "log",
33
"exec": "ping goo.gl"
34
}
35
},
36
"type": "ssh2",
37
"host": "192.168.1.9"
38
},
39
"remotehost": {
40
"parameters": {
41
"user": "muxa",
42
"key": "/home/muxa/.ssh/id_rsa"
43
},
44
"commands": {
45
"pingYa": {
46
"extension": "log",
47
"exec": "ping ya.ru"
48
},
49
"pingGoo": {
50
"extension": "log",
51
"exec": "ping goo.gl"
52
}
53
},
54
"type": "ssh2",
55
"host": "192.168.1.9"
56
},
57
"localhost": {
58
"commands": {
59
"pingYa": {
60
"extension": "log",
61
"exec": "ping ya.ru"
62
},
63
"pingGoo": {
64
"extension": "log",
65
"exec": "ping goo.gl"
66
}
67
},
68
"type": "local"
69
}
70
}
71
}
Copied!
Last modified 9mo ago