Show Posts
1
General / VGC_Analyze - an unofficial analyzer for your VGCollect collection
« on: July 03, 2021, 04:44:50 pm »
Hello everyone,
most of you probably haven’t seen me around. I have been using VGCollect to manage my collection for a number of years, but only recently started using the forum.
I’ve always appreciated the way VGCollect handles collection management but over the years I felt that the analysis side was a little lacking. To determine specific properties of my collection, I often download the CSV-file from my user profile and dig through the data by hand, using a spreadsheet application. I imagine many of you have done the same before.
A while ago I figured I could make my life a little easier by automating the process. So I put together a little script that would download my collection CSV from the website and apply various criteria to analyze it. This turned out to be a great hobby project which provided lots of entertainment in its own right so over the last few months I added a lot of features that I found useful and were fun to implement.
At some point I figured that other users might also be interested in some of the functionality my application provides so I got in touch with the staff to see whether it would be alright to make my project public, some of you might have seen my previous post. Turns out it was alright and so I’d like to introduce you to what I’ve been working on.
VGC_Analyze
The creatively titled VGC_Analyze is an analyzer for your VGCollect game collection.
VGC_Analyze is an unofficial hobby project, not affiliated with VGCollect.com.
For a general overview of the application and installation instructions you can visit the projects GitHub page
In this post I’d like to demonstrate the general functionality and provide some use cases for the application.
Available data / limitations
As stated before, the application uses the CSV backup of your collection as its data source. The CSV backup was initially intended as a way to restore a VGCollect collection on the website and thus only includes data that is necessary for the given task. VGC_Analyze does a lot with the data that is there but some attributes like publisher, developer, genre or release date can’t be evaluated since they are not included in the file.
General mode of operation
The core of VGC_Analyze is the filter sidebar on the left side of the screen. The various filters allow you to pick out specific parts of your collection. All evaluations are always based on the filtered data.
Let’s say I want to compare the number of items of my sixth generation home consoles. I set the platform filter to the respective platforms and all other metrics automatically adjust to the filter result.
Every filter can be combined with every other filter. If you want to see all your japanese import games with missing manuals, purchased between new years and last week, grouped by the month they were added to your collection, you can.
Graphs
Currently VGC_Analyze offers four different graph types. Each graph type can plot multiple attributes like platforms, regions and dates. Some represent total amounts at a specific moment in time, others show growth over a given period.
Templates
Filters can be saved as templates for quick access to often used configurations. The template manager can be accessed via the application menu or via the key combination Ctrl+B.
On the left side the previously saved templates are shown. The entry “[current configuration]” represents the filter attributes that are currently set in the main window and can be saved as a new template. The right side shows the individual attributes of the selected template. Templates can be loaded or deleted.
Collection download
VGC_Analyze can automatically download the CSV backup of your collection from your VGCollect user profile. The download manager can be accessed via the application menu or via the key combination Ctrl+D. The download manager automatically opens on first run when no collection data is present.
Your VGCollect login credentials must be provided to be download your collection. The login credentials are currently not saved and must be re-entered every time you want to update the local copy of your collection. Maybe some time in the future I’ll find the motivation to deal with secure password storage, until then VGC_Analyze forgets credentials immediately.
If you don’t feel like entering your login data into some unknown piece of software (which I totally understand), you can instead download your collection backup manually and copy it into the data folder.
You can access the data folder from the application menu (File -> “Open data folder”).
Restart VGC_Analyze after a new file has been added to the folder. The active file can be selected from the drop down list on the top right of the screen.
Item search
Search through the active list of items with the quick search function, accessible from the application menu or via the key combination Ctrl+F.
Export
VGC_analyze can export your collection as a printable HTML file. The HTML consists of a simple table and reflects the display in the main application one to one. All filters, groupings and sort orders are applied exactly as they are set in the application itself.
The export file is generated in the data folder (File -> “Open data folder”). A simple CSS file is generated alongside the HTML and can be modified if required.
The columns of the HTML table reflect the columns in the main application. Currently there are no separate column configurations for the main table and the export table.
Platform holders
The underlying data for the “platform holder” filter is not provided by the backup CSV file. Instead I chose to implement a list of platform holders that is applied when the data is loaded into the application and can be modified by the user.
To access the platform holders configuration open the application settings (File -> Settings) and select the corresponding settings tab.
VGC_Analyze comes with a default list of platform holders and platform keywords but the list can be modified and expanded by the user.
When a CSV file is read the application checks the platform attributes of each collection item and compares them to the platform keywords. So, as displayed in the above example, any platforms that match “gamecube”, “amiibo” or “wii” are associated with Nintendo while “playstation”, “psone” and “psp” are associated with Sony.
All platforms that can’t be matched to a platform holder are grouped under “[Other]”.
Platform overwrites
To achieve a more consistent data foundation VGC_Analyze provides a function to overwrite platform names. The specific overwrites can be configured in the application settings (File -> Settings -> Platforms).
The above screenshot represents the default list of platform overwrites, which can be modified or expanded by the user.
The reason for platform overwrites is that platforms on VGCollect sometimes have different names for different regions. In general VGC_Analyze tries to extract the region tag of the platform name (e.g. “Nintendo Switch [EU]” is split into platform “Nintendo Switch” and region “EU”) and allows each to be filtered separately.
In the case of platforms that have different names for different regions (e.g. “Nintendo Entertainment System” and “Famicom”) VGC_Analyze can’t make a connection between the two and would list each as its own platform. By overwriting both with “NES/Famicom” the data is combined and can be filtered by region if needed.
Local data extensions
Currently there are two ways to add additional data to your downloaded collection. Items can be marked as bookmarked and/or finished. Both flags are stored locally in the data folder and can either be set via the buttons on the top right of the screen or via the right click context menu.
Regular expressions
The “Title” and “Notes” filters allow the use of regular expressions. To activate regular expressions press the button next to the corresponding input.
VGC_Analyze uses the default Python notation for regular expressions.
For example: “^Legend.*Time$” would match all items that start with “Legend” and end with “Time”.
This concludes the basic overview of VGC_Analyze. Please keep in mind that this is purely a hobby project so some ugly code and a bug or two are to be expected.
If anyone of you is interested in trying the app yourself check out the projects GitHub page. If you have any feedback please post it here as a response.
I’m not a native English speaker so let me know if anything is unclear and I’ll try to describe it in greater detail.
most of you probably haven’t seen me around. I have been using VGCollect to manage my collection for a number of years, but only recently started using the forum.
I’ve always appreciated the way VGCollect handles collection management but over the years I felt that the analysis side was a little lacking. To determine specific properties of my collection, I often download the CSV-file from my user profile and dig through the data by hand, using a spreadsheet application. I imagine many of you have done the same before.
A while ago I figured I could make my life a little easier by automating the process. So I put together a little script that would download my collection CSV from the website and apply various criteria to analyze it. This turned out to be a great hobby project which provided lots of entertainment in its own right so over the last few months I added a lot of features that I found useful and were fun to implement.
At some point I figured that other users might also be interested in some of the functionality my application provides so I got in touch with the staff to see whether it would be alright to make my project public, some of you might have seen my previous post. Turns out it was alright and so I’d like to introduce you to what I’ve been working on.
VGC_Analyze
The creatively titled VGC_Analyze is an analyzer for your VGCollect game collection.
VGC_Analyze is an unofficial hobby project, not affiliated with VGCollect.com.
For a general overview of the application and installation instructions you can visit the projects GitHub page
In this post I’d like to demonstrate the general functionality and provide some use cases for the application.
Available data / limitations
As stated before, the application uses the CSV backup of your collection as its data source. The CSV backup was initially intended as a way to restore a VGCollect collection on the website and thus only includes data that is necessary for the given task. VGC_Analyze does a lot with the data that is there but some attributes like publisher, developer, genre or release date can’t be evaluated since they are not included in the file.
General mode of operation
The core of VGC_Analyze is the filter sidebar on the left side of the screen. The various filters allow you to pick out specific parts of your collection. All evaluations are always based on the filtered data.
Let’s say I want to compare the number of items of my sixth generation home consoles. I set the platform filter to the respective platforms and all other metrics automatically adjust to the filter result.
Every filter can be combined with every other filter. If you want to see all your japanese import games with missing manuals, purchased between new years and last week, grouped by the month they were added to your collection, you can.
Graphs
Currently VGC_Analyze offers four different graph types. Each graph type can plot multiple attributes like platforms, regions and dates. Some represent total amounts at a specific moment in time, others show growth over a given period.
Templates
Filters can be saved as templates for quick access to often used configurations. The template manager can be accessed via the application menu or via the key combination Ctrl+B.
On the left side the previously saved templates are shown. The entry “[current configuration]” represents the filter attributes that are currently set in the main window and can be saved as a new template. The right side shows the individual attributes of the selected template. Templates can be loaded or deleted.
Collection download
VGC_Analyze can automatically download the CSV backup of your collection from your VGCollect user profile. The download manager can be accessed via the application menu or via the key combination Ctrl+D. The download manager automatically opens on first run when no collection data is present.
Your VGCollect login credentials must be provided to be download your collection. The login credentials are currently not saved and must be re-entered every time you want to update the local copy of your collection. Maybe some time in the future I’ll find the motivation to deal with secure password storage, until then VGC_Analyze forgets credentials immediately.
If you don’t feel like entering your login data into some unknown piece of software (which I totally understand), you can instead download your collection backup manually and copy it into the data folder.
You can access the data folder from the application menu (File -> “Open data folder”).
Restart VGC_Analyze after a new file has been added to the folder. The active file can be selected from the drop down list on the top right of the screen.
Item search
Search through the active list of items with the quick search function, accessible from the application menu or via the key combination Ctrl+F.
Export
VGC_analyze can export your collection as a printable HTML file. The HTML consists of a simple table and reflects the display in the main application one to one. All filters, groupings and sort orders are applied exactly as they are set in the application itself.
The export file is generated in the data folder (File -> “Open data folder”). A simple CSS file is generated alongside the HTML and can be modified if required.
The columns of the HTML table reflect the columns in the main application. Currently there are no separate column configurations for the main table and the export table.
Platform holders
The underlying data for the “platform holder” filter is not provided by the backup CSV file. Instead I chose to implement a list of platform holders that is applied when the data is loaded into the application and can be modified by the user.
To access the platform holders configuration open the application settings (File -> Settings) and select the corresponding settings tab.
VGC_Analyze comes with a default list of platform holders and platform keywords but the list can be modified and expanded by the user.
When a CSV file is read the application checks the platform attributes of each collection item and compares them to the platform keywords. So, as displayed in the above example, any platforms that match “gamecube”, “amiibo” or “wii” are associated with Nintendo while “playstation”, “psone” and “psp” are associated with Sony.
All platforms that can’t be matched to a platform holder are grouped under “[Other]”.
Platform overwrites
To achieve a more consistent data foundation VGC_Analyze provides a function to overwrite platform names. The specific overwrites can be configured in the application settings (File -> Settings -> Platforms).
The above screenshot represents the default list of platform overwrites, which can be modified or expanded by the user.
The reason for platform overwrites is that platforms on VGCollect sometimes have different names for different regions. In general VGC_Analyze tries to extract the region tag of the platform name (e.g. “Nintendo Switch [EU]” is split into platform “Nintendo Switch” and region “EU”) and allows each to be filtered separately.
In the case of platforms that have different names for different regions (e.g. “Nintendo Entertainment System” and “Famicom”) VGC_Analyze can’t make a connection between the two and would list each as its own platform. By overwriting both with “NES/Famicom” the data is combined and can be filtered by region if needed.
Local data extensions
Currently there are two ways to add additional data to your downloaded collection. Items can be marked as bookmarked and/or finished. Both flags are stored locally in the data folder and can either be set via the buttons on the top right of the screen or via the right click context menu.
Regular expressions
The “Title” and “Notes” filters allow the use of regular expressions. To activate regular expressions press the button next to the corresponding input.
VGC_Analyze uses the default Python notation for regular expressions.
For example: “^Legend.*Time$” would match all items that start with “Legend” and end with “Time”.
This concludes the basic overview of VGC_Analyze. Please keep in mind that this is purely a hobby project so some ugly code and a bug or two are to be expected.
If anyone of you is interested in trying the app yourself check out the projects GitHub page. If you have any feedback please post it here as a response.
I’m not a native English speaker so let me know if anything is unclear and I’ll try to describe it in greater detail.
