Commandline arguments#2
Open
bozali wants to merge 4 commits into
Open
Conversation
…t so the enduser is able to add BuildSystems through the configuration file
… deserialize to yaml. Also add the CrawlerConfig to the GitHubCrawler
…e, build-system and stars through the commandline
Gismo150
reviewed
Apr 13, 2020
| */ | ||
| private BuildSystem getFileContentsAtRootDir(Repository repository) { | ||
| BuildSystem detectedBuildSystem = BuildSystem.UNKNOWN; | ||
| BuildSystem detectedBuildSystem = BuildSystem.UNKOWN; |
Owner
There was a problem hiding this comment.
Typo in Unkown should be UNKNOWN
Gismo150
reviewed
Apr 13, 2020
Comment on lines
+14
to
+16
| public static final String FILEPATH = null;//PropertyFileReader.getInstance().getProperty(EConfig.FILEPATH); | ||
| public static final String JSONFILENAME = null;//"repositories.json"; | ||
| public static final String CUSTOMFILE = null;//PropertyFileReader.getInstance().getProperty(EConfig.CUSTOMFILE); |
Owner
There was a problem hiding this comment.
These properties are never set by the configservice.
Read the comment on the ConfigurationService file.
The FileHelper.java file then needs to be updated to get the configs from the CrawlerConfig and not the old Config file.
Gismo150
requested changes
Apr 13, 2020
Gismo150
left a comment
Gismo150
left a comment
Owner
There was a problem hiding this comment.
Please read the comments within the commited files.
| * | ||
| * @return Returns the @see CrawlerConfig. | ||
| */ | ||
| public CrawlerConfig getConfig() { |
Owner
There was a problem hiding this comment.
We need another function that we can call to simply get the already parsed Configuration.
This service should only parse the yaml file once and provide the CrawlerConfig with a getter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It is now possible to use the crawler through the command-line e.g.
crawler.jar crawl --language="CPP" --build-system="CUSTOM" --stars=1
If you execute the jar file without any parameters it will load the configuration file.
It now supports now sub-commands but the only sub-command that is added is "crawl" with three parameters.
Also the configuration file was moved to the %USERDATA%/crawler/config.yaml for easier management (de-/serialization). Along the configuration changes some other models have been recreated like BuildSystem.java for the Yaml serializer. With the changes it is now possible to define custom Build-Systems inside the configuration.
WARNING:
There are some configuration items that needs to be changed. Not everything was changed with this pull request. There are some util classes that still use the empty static config file like FileHelper.getRepositoriesJsonFilePath. The static config file must be fully replaced by the CrawlerConfig.
Also I am not sure if my changes are working 100% like before this needs to be tested and reviewed!