Cannot accomplish a "make all" command

[jukito]

Hi.

When i'm trying to run the make all command, I get this:
[root@localhost cmkbeat]# make all
make deps
make[1]: Entering directory /root/wok/cmkbeat
glide up
[INFO] Downloading dependencies. Please wait...
[INFO] --> Fetching updates for github.com/elastic/beats
[INFO] --> Fetching updates for github.com/jeremyweader/go-livestatus
[INFO] Resolving imports
[INFO] --> Fetching updates for github.com/jeremyweader/cmkbeat
[INFO] --> Fetching updates for github.com/satori/go.uuid
[INFO] --> Fetching updates for github.com/elastic/go-ucfg
[INFO] --> Fetching updates for github.com/pkg/errors
[INFO] --> Fetching updates for go.uber.org/zap
[INFO] --> Fetching updates for golang.org/x/sys
[INFO] --> Fetching updates for github.com/spf13/cobra
[INFO] --> Fetching updates for github.com/spf13/pflag
[INFO] --> Fetching updates for golang.org/x/crypto
[ERROR] Error scanning github.com/jeremyweader/cmkbeat/cmd: cannot find package "." in:
/root/.glide/cache/src/https-github.com-jeremyweader-cmkbeat/cmd
[INFO] --> Fetching updates for gopkg.in/yaml.v2
[INFO] --> Fetching updates for go.uber.org/atomic
[INFO] --> Detected semantic version. Setting version for go.uber.org/atomic to v1.3.2
[INFO] --> Fetching updates for go.uber.org/multierr
[INFO] --> Detected semantic version. Setting version for go.uber.org/multierr to v1.1.0
[INFO] --> Fetching updates for github.com/joeshaw/multierror
[INFO] --> Fetching updates for github.com/mitchellh/hashstructure
[INFO] --> Fetching updates for github.com/elastic/go-sysinfo
[INFO] --> Fetching updates for github.com/inconshreveable/mousetrap
[INFO] --> Fetching updates for github.com/elastic/go-seccomp-bpf
[INFO] --> Fetching updates for github.com/elastic/gosigar
[INFO] --> Fetching updates for github.com/elastic/go-structform
[INFO] --> Fetching updates for github.com/fatih/color
[INFO] --> Fetching updates for github.com/mattn/go-colorable
[INFO] --> Fetching updates for golang.org/x/net
[INFO] --> Fetching updates for github.com/docker/docker
[INFO] --> Fetching updates for github.com/docker/go-connections
[INFO] --> Fetching updates for github.com/ericchiang/k8s
[INFO] --> Fetching updates for github.com/ghodss/yaml
[INFO] --> Fetching updates for github.com/StackExchange/wmi
[INFO] --> Fetching updates for github.com/Shopify/sarama
[INFO] --> Fetching updates for github.com/rcrowley/go-metrics
[INFO] --> Fetching updates for github.com/elastic/go-lumber
[INFO] --> Fetching updates for github.com/garyburd/redigo
[INFO] --> Fetching updates for github.com/dustin/go-humanize
[INFO] --> Fetching updates for github.com/elastic/go-txfile
[INFO] --> Fetching howett.net/plist
[WARN] Unable to checkout howett.net/plist
[ERROR] Error looking for howett.net/plist: Unable to get repository: Cloning into '/root/.glide/cache/src/https-howett.net-plist'...
error: RPC failed; result=22, HTTP code = 404
fatal: The remote end hung up unexpectedly
: exit status 128
[INFO] --> Fetching updates for github.com/prometheus/procfs
[INFO] --> Fetching updates for github.com/elastic/go-windows
[INFO] --> Fetching updates for github.com/mattn/go-isatty
[INFO] --> Fetching updates for github.com/docker/go-units
[INFO] --> Fetching updates for github.com/docker/distribution
[INFO] --> Fetching updates for github.com/opencontainers/go-digest
[INFO] --> Fetching updates for github.com/golang/protobuf
[INFO] --> Fetching updates for github.com/go-ole/go-ole
[INFO] --> Fetching updates for github.com/davecgh/go-spew
[INFO] --> Fetching updates for github.com/eapache/go-resiliency
[INFO] --> Fetching updates for github.com/eapache/go-xerial-snappy
[INFO] --> Fetching updates for github.com/eapache/queue
[INFO] --> Fetching updates for github.com/pierrec/lz4
[INFO] --> Fetching updates for github.com/klauspost/compress
[INFO] --> Fetching updates for github.com/urso/go-bin
[INFO] --> Fetching updates for github.com/opencontainers/image-spec
[INFO] --> Fetching updates for github.com/Microsoft/go-winio
[INFO] --> Fetching updates for github.com/golang/snappy
[INFO] --> Fetching updates for github.com/theckman/go-flock
[INFO] --> Fetching updates for github.com/gogo/protobuf
[INFO] --> Fetching updates for golang.org/x/text
[INFO] --> Fetching updates for github.com/klauspost/cpuid
[ERROR] Failed to retrieve a list of dependencies: Error resolving imports
make[1]: *** [deps] Error 1
make[1]: Leaving directory `/root/wok/cmkbeat'
make: *** [all] Error 2

Greetings

[jukito]

I resolve the first error simply creating the folder "/root/.glide/cache/src/https-github.com-jeremyweader-cmkbeat/cmd"

The second error is an 404 from the howett.net/plist web and I'm not able to find where is the reference to this web to try to change it.

[moesch]

Also not able to compile this. Create folder https-github.com-jeremyweader-cmkbeat/cmd did not solve the problem. I'm not very familiar with Go. So I didn't not really understood what is missing here.

Thomas

[jvanhoose]

Ok, I was finally able to make this thing compile - but it has some pretty major, shall we say, challenges....

First, unlike what it says in the instructions, I was not able to make it compile on go version 1.7 - I ended up accomplishing it on version 1.10. Second, the directory and path has to be in a very specific place under the GOPATH. So, here is what I did (copied from history - so a few things might be unnecessary, but it is what it is):

export GOPATH=/usr/local/go
apt-get install golang-1.10-go
export PATH=$PATH:/usr/lib/go-1.10/bin
mkdir -p $GOPATH/src/github.com/jeremyweader
cd $GOPATH/src/github.com/jeremyweader
git clone https://github.com/comnetgmbh/cmkbeat.git
go get github.com/Masterminds/glide

On my system (Ubuntu 16 - fairly stock), one of the dependencies didn't work, so I had to:

glide mirror set https://golang.org/x/crypto https://github.com/golang/crypto --vcs git

I also had to:

apt-get install python-yaml

After that, it was able to compile successfully:

'# make all
make deps
make[1]: Entering directory '/usr/local/go/src/github.com/jeremyweader/cmkbeat'
glide up
[WARN] The name listed in the config file (.) does not match the current location (github.com/jeremyweader/cmkbeat)
[INFO] Loading mirrors from mirrors.yaml file
[INFO] Downloading dependencies. Please wait...
[INFO] --> Fetching updates for github.com/jeremyweader/go-livestatus
[INFO] --> Fetching updates for github.com/elastic/beats
[INFO] Resolving imports
[INFO] --> Fetching updates for github.com/gofrs/uuid
[INFO] --> Fetching updates for github.com/elastic/go-ucfg
[INFO] --> Fetching updates for github.com/pkg/errors
[INFO] --> Fetching updates for go.uber.org/zap
[INFO] --> Fetching updates for golang.org/x/sys
[INFO] --> Fetching updates for github.com/spf13/cobra
[INFO] --> Fetching updates for github.com/spf13/pflag
[INFO] --> Fetching updates for golang.org/x/crypto
[INFO] --> Fetching updates for gopkg.in/yaml.v2
[INFO] --> Fetching updates for go.uber.org/atomic
[INFO] --> Detected semantic version. Setting version for go.uber.org/atomic to v1.3.2
[INFO] --> Fetching updates for go.uber.org/multierr
[INFO] --> Detected semantic version. Setting version for go.uber.org/multierr to v1.1.0
[INFO] --> Fetching updates for github.com/joeshaw/multierror
[INFO] --> Fetching updates for github.com/mitchellh/hashstructure
[INFO] --> Fetching updates for github.com/elastic/go-sysinfo
[INFO] --> Fetching updates for github.com/inconshreveable/mousetrap
[INFO] --> Fetching updates for github.com/elastic/go-seccomp-bpf
[INFO] --> Fetching updates for github.com/elastic/gosigar
[INFO] --> Fetching updates for github.com/elastic/go-structform
[INFO] --> Fetching updates for github.com/miekg/dns
[INFO] --> Fetching updates for github.com/rcrowley/go-metrics
[INFO] --> Fetching updates for github.com/fatih/color
[INFO] --> Fetching updates for github.com/mattn/go-colorable
[INFO] --> Fetching updates for golang.org/x/net
[INFO] --> Fetching updates for github.com/docker/docker
[INFO] --> Fetching updates for github.com/docker/go-connections
[INFO] --> Fetching updates for github.com/ericchiang/k8s
[INFO] --> Fetching updates for github.com/ghodss/yaml
[INFO] --> Fetching updates for github.com/StackExchange/wmi
[INFO] --> Fetching updates for github.com/Shopify/sarama
[INFO] --> Fetching updates for github.com/elastic/go-lumber
[INFO] --> Fetching updates for github.com/garyburd/redigo
[INFO] --> Fetching updates for github.com/dustin/go-humanize
[INFO] --> Fetching updates for github.com/elastic/go-txfile
[INFO] --> Fetching updates for howett.net/plist
[INFO] --> Fetching updates for github.com/prometheus/procfs
[INFO] --> Fetching updates for github.com/elastic/go-windows
[INFO] --> Fetching updates for github.com/mattn/go-isatty
[INFO] --> Fetching updates for github.com/docker/go-units
[INFO] --> Fetching updates for github.com/docker/distribution
[INFO] --> Fetching updates for github.com/opencontainers/go-digest
[INFO] --> Fetching updates for github.com/golang/protobuf
[INFO] --> Fetching updates for github.com/go-ole/go-ole
[INFO] --> Fetching updates for github.com/DataDog/zstd
[INFO] --> Fetching updates for github.com/davecgh/go-spew
[INFO] --> Fetching updates for github.com/eapache/go-resiliency
[INFO] --> Fetching updates for github.com/eapache/go-xerial-snappy
[INFO] --> Fetching updates for github.com/eapache/queue
[INFO] --> Fetching updates for github.com/pierrec/lz4
[INFO] --> Fetching updates for github.com/klauspost/compress
[INFO] --> Fetching updates for github.com/urso/go-bin
[INFO] --> Fetching updates for github.com/opencontainers/image-spec
[INFO] --> Fetching updates for github.com/Microsoft/go-winio
[INFO] --> Fetching updates for github.com/golang/snappy
[INFO] --> Fetching updates for github.com/theckman/go-flock
[INFO] --> Fetching updates for github.com/gogo/protobuf
[INFO] --> Fetching updates for golang.org/x/text
[INFO] --> Fetching updates for github.com/klauspost/cpuid
[INFO] Downloading dependencies. Please wait...
[INFO] Setting references for remaining imports
[INFO] Exporting resolved dependencies...
[INFO] --> Exporting github.com/jeremyweader/go-livestatus
[INFO] --> Exporting github.com/elastic/go-ucfg
[INFO] --> Exporting github.com/pkg/errors
[INFO] --> Exporting github.com/elastic/go-structform
[INFO] --> Exporting github.com/elastic/beats
[INFO] --> Exporting github.com/spf13/cobra
[INFO] --> Exporting github.com/rcrowley/go-metrics
[INFO] --> Exporting github.com/miekg/dns
[INFO] --> Exporting github.com/gofrs/uuid
[INFO] --> Exporting go.uber.org/zap
[INFO] --> Exporting github.com/spf13/pflag
[INFO] --> Exporting github.com/elastic/gosigar
[INFO] --> Exporting github.com/elastic/go-sysinfo
[INFO] --> Exporting golang.org/x/crypto
[INFO] --> Exporting github.com/mitchellh/hashstructure
[INFO] --> Exporting github.com/joeshaw/multierror
[INFO] --> Exporting github.com/inconshreveable/mousetrap
[INFO] --> Exporting github.com/elastic/go-seccomp-bpf
[INFO] --> Exporting golang.org/x/sys
[INFO] --> Exporting go.uber.org/multierr
[INFO] --> Exporting go.uber.org/atomic
[INFO] --> Exporting github.com/fatih/color
[INFO] --> Exporting github.com/mattn/go-colorable
[INFO] --> Exporting github.com/docker/docker
[INFO] --> Exporting github.com/docker/go-connections
[INFO] --> Exporting github.com/ghodss/yaml
[INFO] --> Exporting github.com/ericchiang/k8s
[INFO] --> Exporting github.com/Shopify/sarama
[INFO] --> Exporting github.com/StackExchange/wmi
[INFO] --> Exporting github.com/dustin/go-humanize
[INFO] --> Exporting github.com/garyburd/redigo
[INFO] --> Exporting github.com/elastic/go-lumber
[INFO] --> Exporting github.com/elastic/go-txfile
[INFO] --> Exporting golang.org/x/net
[INFO] --> Exporting github.com/prometheus/procfs
[INFO] --> Exporting github.com/elastic/go-windows
[INFO] --> Exporting github.com/mattn/go-isatty
[INFO] --> Exporting github.com/docker/go-units
[INFO] --> Exporting github.com/docker/distribution
[INFO] --> Exporting github.com/opencontainers/go-digest
[INFO] --> Exporting github.com/golang/protobuf
[INFO] --> Exporting github.com/go-ole/go-ole
[INFO] --> Exporting github.com/DataDog/zstd
[INFO] --> Exporting github.com/davecgh/go-spew
[INFO] --> Exporting github.com/eapache/go-resiliency
[INFO] --> Exporting github.com/eapache/go-xerial-snappy
[INFO] --> Exporting howett.net/plist
[INFO] --> Exporting github.com/pierrec/lz4
[INFO] --> Exporting github.com/eapache/queue
[INFO] --> Exporting github.com/klauspost/compress
[INFO] --> Exporting github.com/urso/go-bin
[INFO] --> Exporting github.com/opencontainers/image-spec
[INFO] --> Exporting github.com/Microsoft/go-winio
[INFO] --> Exporting gopkg.in/yaml.v2
[INFO] --> Exporting github.com/gogo/protobuf
[INFO] --> Exporting github.com/theckman/go-flock
[INFO] --> Exporting github.com/golang/snappy
[INFO] --> Exporting github.com/klauspost/cpuid
[INFO] --> Exporting golang.org/x/text
[INFO] Replacing existing vendor dependencies
[INFO] Versions did not change. Skipping glide.lock update.
[INFO] Project relies on 59 dependencies.
make[1]: Leaving directory '/usr/local/go/src/github.com/jeremyweader/cmkbeat'
make cmkbeat
make[1]: Entering directory '/usr/local/go/src/github.com/jeremyweader/cmkbeat'
go build -i -ldflags "-X github.com/elastic/beats/libbeat/version.buildTime=2018-12-20T22:15:54Z -X github.com/elastic/beats/libbeat/version.commit=9da31afd65a3c9bb8e9bf2f7a9f1a9b56bee278d"
make[1]: Leaving directory '/usr/local/go/src/github.com/jeremyweader/cmkbeat'

After that, when I tried the install:

'# make install
mkdir -p /etc/cmkbeat
mkdir -p /usr/share/cmkbeat/bin
mkdir -p /var/lib/cmkbeat
mkdir -p /var/log/cmkbeat
cp cmkbeat /usr/share/cmkbeat/bin/.
cp *.yml /etc/cmkbeat/.
cp *.json /etc/cmkbeat/.
cp system/cmkbeat.service /usr/lib/systemd/system/.
cp: cannot create regular file '/usr/lib/systemd/system/.': No such file or directory
Makefile:40: recipe for target 'install' failed
make: *** [install] Error 1

Might be an ubuntu 16 thing... but a:

mkdir -p /usr/lib/systemd/system/

'# make install
mkdir -p /etc/cmkbeat
mkdir -p /usr/share/cmkbeat/bin
mkdir -p /var/lib/cmkbeat
mkdir -p /var/log/cmkbeat
cp cmkbeat /usr/share/cmkbeat/bin/.
cp *.yml /etc/cmkbeat/.
cp *.json /etc/cmkbeat/.
cp system/cmkbeat.service /usr/lib/systemd/system/.
systemctl enable cmkbeat.service
Created symlink from /etc/systemd/system/multi-user.target.wants/cmkbeat.service to /usr/lib/systemd/system/cmkbeat.service.

So, yes, it can compile and install... haven't got it actually running (just haven't tried yet), but it can be compiled with a bit of care and feeding.

YMMV.

[jvanhoose]

Ok, I finally got it actually moving data into elasticsearch... had to do a few tweaks to make it work:

1. By default, when it compiled, it compiled with version 7.0.0 of the beats code. Apparently, there are things that have changed on that side, and I was getting a weird error:

2018-12-20T16:51:45.121-0600 WARN elasticsearch/client.go:531 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbeff251da61cf3f5, ext:30253169745, loc:(*time.Location)(0x2051780)}, Meta:common.MapStr(nil), Fields:common.MapStr{"display_name":"Kernel Process Creations", "plugin_output":"OK - 3/s", "percent_state_change":"0", "perf_data":"processes=3.1;;;;", "type":"its-nagios-p-integrate-01", "state":"0", "long_plugin_output":"", "host_name":"ome-lti-p2-app-01", "host":common.MapStr{"name":"its-nagios-p-integrate-01"}, "agent":common.MapStr{"type":"cmkbeat", "hostname":"its-nagios-p-integrate-01", "version":"7.0.0"}, "metrics":common.MapStr{"Kernel_Process_Creations":common.MapStr{"processes":"3.1"}}}, Private:interface {}(nil)}, Flags:0x0} (status=400): {"type":"invalid_type_name_exception","reason":"Document mapping type name can't start with '_', found: [_doc]"}

Basically, found a tech note that indicates that this is specific to 7.0.0 (hope they fix it before they screw things up again). We are running elasticsearch 5.6.3 - so, this no workie.

To fix it, I changed the glide.yml file to specify a slightly older version:

package: .
import:

• package: github.com/elastic/beats
  version: ^6.x
  subpackages:
  • libbeat/beat
  • libbeat/common
  • libbeat/logp
  • libbeat/publisher
• package: github.com/jeremyweader/go-livestatus

The version code is the newer element. After a recompile and reinstall, it seemed to start generating data. Haven't played around with it as yet to see what its actually dumping, but at least, it appears to be doing something:

2018-12-20T17:22:17.262-0600 INFO beater/cmkbeat.go:258 11398 events submitted in 14.311433679s.
2018-12-20T17:22:49.530-0600 INFO beater/cmkbeat.go:258 11398 events submitted in 16.579886007s.

So, there is definitely some smoke... now, to see if there is any fire.

Jeff :-)

[dhanumjaya]

Currently we are using check_mk 1.5 version, elasticsearch 6.4.0 version. Requesting you to suggest cmkbeat version need to configure to fetch the data form check-mk to elasticsearch.

currently we cmkbeat verison is 6.6.0 and we are not able to fetch the services data using livestatus feature. please suggest.