PMM-5680 store agents logs by qwest812 · Pull Request #197 · percona/pmm-admin

qwest812 · 2022-04-02T15:29:30Z

PMM-5680 store logs pmm-agent#342

BupycHuk · 2022-04-12T22:30:02Z

@qwest812 please make PR is ready for review if it's ready

# Conflicts: # go.mod # go.sum

github-actions · 2022-04-13T21:07:41Z

+			logrus.Debugf("%s", err)
+			b = []byte(err.Error())
+		}
+		b = append(b, '\n')


🚫 [golangci-lint] _{reported by reviewdog 🐶}
append only allowed to cuddle with appended value (wsl)

BupycHuk · 2022-04-18T23:36:05Z

+			b = []byte(err.Error())
+		}
+		b = append(b, '\n')
+		addData(zipW, "logs/"+pointer.GetString(agent.AgentType)+".json", now, bytes.NewReader(b))


what is the reason to make it json?
I think it would be much easier to read if it would be just a text with some separator.
something like

agent id: /agent_id/xxxx1 agent type: mongodb_exporter lorem ipsum log texts begin ... lorem ipsum log texts =============================================== agent id: /agent_id/xxxx2 agent type: mysqld_exporter lorem ipsum log texts begin ... lorem ipsum log texts

# Conflicts: # go.mod # go.sum

BupycHuk · 2022-05-03T14:29:09Z

+	fileName := "pmm-admin-summary.zip"
+	err = downloadFile(fmt.Sprintf("http://%s:%d/logs.zip", agentlocal.Localhost, agentlocal.DefaultPMMAgentListenPort), fileName)
+	if err != nil {
+		logrus.Debugf("%s", err)
+		b = []byte(err.Error())
+	}
+	logrus.Info("pmm-admin-summary.zip created")
+


we are on a right track, as a next step let's include files from logs.zip into main zip file

you can find example of it in addServerData

BupycHuk

LGTM, a few minor changes

BupycHuk · 2022-05-04T10:12:35Z


 	addData(zipW, "client/pmm-admin-version.txt", now, bytes.NewReader([]byte(version.FullInfo())))

+	err = downloadFile(zipW, fmt.Sprintf("http://%s:%d/logs.zip", agentlocal.Localhost, agentlocal.DefaultPMMAgentListenPort), "pmm-admin logs")


Suggested change

err = downloadFile(zipW, fmt.Sprintf("http://%s:%d/logs.zip", agentlocal.Localhost, agentlocal.DefaultPMMAgentListenPort), "pmm-admin logs")

err = downloadFile(zipW, fmt.Sprintf("http://%s:%d/logs.zip", agentlocal.Localhost, agentlocal.DefaultPMMAgentListenPort), "pmm-agent")

BupycHuk · 2022-05-04T10:13:47Z


+	err = downloadFile(zipW, fmt.Sprintf("http://%s:%d/logs.zip", agentlocal.Localhost, agentlocal.DefaultPMMAgentListenPort), "pmm-admin logs")
+	if err != nil {
+		logrus.Debugf("%s", err)


let's write it in warn level

BupycHuk · 2022-05-04T10:14:30Z

 	return b, nil
 }

+// downloadFile download file to local destination


Suggested change

// downloadFile download file to local destination

// downloadFile download file and includes into zip file

BupycHuk · 2022-05-04T10:15:26Z


+// downloadFile download file to local destination
+func downloadFile(zipW *zip.Writer, url, fileName string) error {
+


please remove this empty line, it makes CI fail

github-actions · 2022-05-04T10:24:05Z

+	if err != nil {
+		return err
+	}
+	defer response.Body.Close()


🚫 [golangci-lint] _{reported by reviewdog 🐶}
Error return value of response.Body.Close is not checked (errcheck)

github-actions · 2022-05-04T10:24:05Z

+		}
+		addData(zipW, path.Join(fileName, rf.Name), rf.Modified, rc)
+
+		rc.Close()


🚫 [golangci-lint] _{reported by reviewdog 🐶}
Error return value of rc.Close is not checked (errcheck)

github-actions · 2022-05-04T10:24:05Z


+// downloadFile download file and includes into zip file
+func downloadFile(zipW *zip.Writer, url, fileName string) error {
+	response, err := http.Get(url)


🚫 [golangci-lint] _{reported by reviewdog 🐶}
G107: Potential HTTP request made with variable url (gosec)

github-actions · 2022-05-04T10:24:05Z

+			logrus.Errorf("%s", err)
+			continue
+		}
+		addData(zipW, path.Join(fileName, rf.Name), rf.Modified, rc)


🚫 [golangci-lint] _{reported by reviewdog 🐶}
G305: File traversal when extracting zip/tar archive (gosec)

github-actions · 2022-05-04T10:24:05Z

+	}
+	defer response.Body.Close()
+
+	if response.StatusCode != 200 {


🚫 [golangci-lint] _{reported by reviewdog 🐶}
mnd: Magic number: 200, in detected (gomnd)

github-actions · 2022-05-04T10:24:05Z

+		rc, err := rf.Open()
+		if err != nil {
+			logrus.Errorf("%s", err)
+			continue


🚫 [golangci-lint] _{reported by reviewdog 🐶}
continue with no blank line before (nlreturn)

github-actions · 2022-05-04T10:24:05Z

+func downloadFile(zipW *zip.Writer, url, fileName string) error {
+	response, err := http.Get(url)
+	if err != nil {
+		return err


🚫 [golangci-lint] _{reported by reviewdog 🐶}
error returned from external package is unwrapped: sig: func net/http.Get(url string) (resp *net/http.Response, err error) (wrapcheck)

PMM-5680 store agents logs

3bc9d37

qwest812 requested a review from BupycHuk as a code owner April 2, 2022 15:29

qwest812 marked this pull request as draft April 2, 2022 15:33

This was referenced Apr 2, 2022

PMM-5680 store logs Percona-Lab/pmm-submodules#2425

Closed

PMM-5680 store logs percona/pmm-agent#342

Open

PMM-5680 add log to struct AgentInfo percona/pmm#857

Closed

YaroslavPodorvanov approved these changes Apr 12, 2022

View reviewed changes

qwest812 mentioned this pull request Apr 13, 2022

PMM-5680 add log to struct AgentInfo percona/pmm#866

Closed

2 tasks

qwest812 added 2 commits April 14, 2022 00:04

Merge branch 'main' into PMM-5680-store-logs

2db5722

# Conflicts: # go.mod # go.sum

PMM-5680 store agents logs

e1df266

github-actions Bot reviewed Apr 13, 2022

View reviewed changes

qwest812 added 2 commits April 14, 2022 00:13

PMM-5680 update dependency

1fc2698

PMM-5680 update dependency

df0f287

qwest812 mentioned this pull request Apr 14, 2022

PMM-5680 store logs percona/pmm#868

Closed

2 tasks

qwest812 marked this pull request as ready for review April 14, 2022 21:20

YaroslavPodorvanov approved these changes Apr 14, 2022

View reviewed changes

BupycHuk reviewed Apr 18, 2022

View reviewed changes

floatinginbits self-requested a review April 21, 2022 10:45

qwest812 added 5 commits April 30, 2022 09:50

Merge branch 'main' into PMM-5680-store-logs

65cb23f

# Conflicts: # go.mod # go.sum

PMM-5680 create zip file with pmm-agent logs

1a9381c

Merge branch 'main' into PMM-5680-store-logs

4a86454

# Conflicts: # go.mod # go.sum

PMM-5680 pretty

ed34aaf

PMM-5680 remove TODO

1e5e1ab

qwest812 requested review from BupycHuk and YaroslavPodorvanov May 3, 2022 11:04

YaroslavPodorvanov approved these changes May 3, 2022

View reviewed changes

BupycHuk reviewed May 3, 2022

View reviewed changes

qwest812 added 3 commits May 3, 2022 22:28

PMM-5680 save logs to file zip

3d9236e

PMM-5680 pretty

253991c

change request

3b5020a

qwest812 requested a review from BupycHuk May 4, 2022 09:44

BupycHuk approved these changes May 4, 2022

View reviewed changes

BupycHuk reviewed May 4, 2022

View reviewed changes

fix bug

e23d761

github-actions Bot reviewed May 4, 2022

View reviewed changes

Merge branch 'main' into PMM-5680-store-logs

f69bd1e


		addData(zipW, "client/pmm-admin-version.txt", now, bytes.NewReader([]byte(version.FullInfo())))

		err = downloadFile(zipW, fmt.Sprintf("http://%s:%d/logs.zip", agentlocal.Localhost, agentlocal.DefaultPMMAgentListenPort), "pmm-admin logs")

	// downloadFile download file to local destination
	// downloadFile download file and includes into zip file


		// downloadFile download file to local destination
		func downloadFile(zipW *zip.Writer, url, fileName string) error {

Conversation

qwest812 commented Apr 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BupycHuk commented Apr 12, 2022

Uh oh!

github-actions Bot Apr 13, 2022

Choose a reason for hiding this comment

Uh oh!

BupycHuk Apr 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BupycHuk May 3, 2022

Choose a reason for hiding this comment

Uh oh!

BupycHuk May 3, 2022

Choose a reason for hiding this comment

Uh oh!

qwest812 May 4, 2022

Choose a reason for hiding this comment

Uh oh!

BupycHuk left a comment

Choose a reason for hiding this comment

Uh oh!

BupycHuk May 4, 2022

Choose a reason for hiding this comment

Uh oh!

BupycHuk May 4, 2022

Choose a reason for hiding this comment

Uh oh!

BupycHuk May 4, 2022

Choose a reason for hiding this comment

Uh oh!

BupycHuk May 4, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions Bot May 4, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions Bot May 4, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions Bot May 4, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions Bot May 4, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions Bot May 4, 2022

Choose a reason for hiding this comment

Uh oh!

github-actions Bot May 4, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions Bot May 4, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

qwest812 commented Apr 2, 2022 •

edited

Loading

BupycHuk Apr 18, 2022 •

edited

Loading