SiteAnalyzer - free SEO spider and audit tool

Started by majento, 11-22-2017, 06:00:11

Previous topic - Next topic

majentoTopic starter

Dear colleagues!

I present to you the SiteAnalyzer program, which is intended for technical analysis and site audit (server response codes, presence of "broken" links, links to external resources, check for availability of title, keywords, description, h1-h6 headers, check for duplicate pages, etc). This is a free analogue of the program Screaming Frog SEO Spider and similar to it.

The analysis is performed by scanning all pages of the site (crawling the site by links), after which, based on parsed data, you can audit the parameters of interest.





Key features

  • Scanning of all pages of the site, as well as images, scripts and documents
  • Obtaining server response codes for each page (200, 301, 302, 404, 500, 503, etc.)
  • Determining the presence and content Title, Keywords, Description, H1-H6
  • Search and display the "duplicate" pages, meta tags and headers
  • Determining whether an attribute rel="canonical" for each page of the site
  • Following the directives of "robots.txt" file or meta tag "robots"
  • Accounting rel="nofollow" when crawling pages on your site
  • Reference analysis – finding internal and external links to pages (within the site)
  • Determination of the number of referrals from the page (redirect)
  • Determining the level of nesting pages relative to the main
  • Generate sitemap "sitemap.xml" (with the possibility of splitting into several files)
  • URL filtering by any parameter
  • Export reports to CSV and Excel (full report in Excel-format)
Differences from analogues

  • Low demands on computer resources, low consumption of RAM
  • To store data, a local database is used that is characterized by its performance and reliability
  • Scanning websites of any volumes due to the low requirements of computer resources
  • Portable format (works without installation on a PC or directly from removable media)
I will be glad to any comments and wishes on the work and development of the program.

Sorry for my bad English :(

Program page: https://site-analyzer.pro/
Download: majento-siteanalyzer.zip (2,5 Mb)

PS. The program is distributed free (there is a Buy item on the site, but in the near future payment for the program will not be charged).
  •  


majentoTopic starter

New version SiteAnalyzer 1.4.2.85





What's new:

  • added limit the number of pages scanned
  • added the ability to scan selected projects (scan several projects in turn)
  • added the ability to delete selected projects
Web: SiteAnalyzer (2,5 Mb)
  •  


majentoTopic starter

New version SiteAnalyzer 1.4.3.89



What's new:

  • it became possible to add sites to the list of projects
  • added display of data download speed when parsing sites (URI/sec)
  • added showing of favicon sites in the list of projects
  • restored the display of data in the "Info" tab
Web: majento-siteanalyzer.zip (2,5 Mb)
  •  

majentoTopic starter

New version SiteAnalyzer 1.5.0.101







What's new:

  • increased the speed of parsing websites, increased stability of the program
  • added the ability to download reports in Excel format (all results in one file, with tabs)
  • added parsing video (avi, mpeg, mov, links to Youtube, etc. formats). New column "Video" in the main report + tab
  • added parsing of documents (doc, xlsx, pdf, zip, mp3, etc. formats). New column "Documents" in the main report + tab
  • added highlighting of lines around content types (URL, styles, scripts, images, videos, documents, etc.)
  • added a visual display of the progress of data loading for "large" projects
  • restored parsing and displaying files in *.css format
  • corrected incorrect accounting of the rules of the file "robots.txt" when parsing websites
  • optimized database structure, the removal of projects began to occur many times faster
Website: SiteAnalyzer (3 Mb)
  •  

majentoTopic starter

New version SiteAnalyzer 1.5.1.104





What's new:

  • increased speed of data export to Excel + eliminated bugs that occur when exporting large projects
  • increased speed of generating a sitemap "Sitemap.xml"
  • setting up the program for the "Scan" and "Exceptions" tabs became individual for each project
  • fixed a bug where objects with the type "text/javascript" were recognized as documents, rather than files like "Javascript"
  • fixed incorrect parsing of the meta-tag TITLE with the SVG-formatted image on the page (in which this tag is also present)
  • fixed incorrect sorting of the main site data in the table, boolean type output (for example, sorting in the HTTPS column)
  • added a pie chart showing the percentage of objects on the site (pages, images, styles, scripts, etc.)
Web: SiteAnalyzer (3 Mb)
  •  


virtuatechnologies

Site-Analyzer, the most efficient Website Analyzer which means such as Crawl, On-page Analysis, Rank Tracking, Traffic Checker, Backlinks Checker and much more...

majentoTopic starter

New version SiteAnalyzer 2.0







What's new:

  • added website structure visualization on a graph (similar to Screaming Frog)
  • added a feature to crawl local websites by URLs, not just by the main page, as it was before
  • added a feature to crawl local websites running on Live Server (and other servers) at URLs like http://127.0.0.1:5500/
  • added automatic re-crawling of pages if their loading is timed out (for example, when the website blocks a large number of simultaneous requests and returns undefined response codes)
  • added automatic update of program versions (at program startup or through the main menu)
  • added the option to copy text from the URL entry field by Ctrl-C
  • added counter for the remaining time of the current project crawling
  • many other changes...
More info: SiteAnalyzer 2.0
  •  

majentoTopic starter

New version SiteAnalyzer 2.0.2







What's new:

  • increased crawling speed (growth of about 10%), increased stability of the scanner
  • the speed of drawing the nodes of the graph is increased by an order of 10 times (now the graph can easily display up to 100,000 nodes)
  • optimized time for the final saving of the project to the database (relevant for large sites)
  • optimized display of duplicate headers H1-H6 (duplicate pages removed)
  • added the ability to turn off alerts about the release of a new version when the program starts
  • added a column indicating the number of incoming links to a page on the PageRank tab
  • added ability to rotate the graph and change the scale by an arbitrary value
  • fixed not always correct definition of non-canonical pages (Wrong Canonical)
  • «Access violation» error occurred when deleting a large number of URLs
  • URL disappearance bug fixed when pausing the project / URL list crawl process
  • fixed a bug where the Delete button did not work in the Quick filtering field
  • minor usability errors fixed, minor bugs fixed
More info: SiteAnalyzer 2.0.2
  •  


majentoTopic starter

New version SiteAnalyzer 2.1







What's new:

  • added the ability to export every external link, 404 error, and image in a single report together with all the pages that contain them
  • added the ability to manually rescan the URLs that return a response code of 0 (Read Timeout error) in one click from the context menu
  • added the ability to hide and display certain columns and tabs in the main data block
  • added the "Read Timeout" and "Blocked by Robots.txt" status updates for the URLs that return a status code of 0
  • added a display mode for the graph visualization that sets the size of the nodes based on their PageRank
  • added an extra button to calculate PageRank on the corresponding tab
  • added the ability to rearrange and save the order of columns on tabs
  • the active tab is now saved when switching between projects
  • added the ability to parse the URLs protected with a .htpasswd file
  • added the ability to export the data from the Dashboard tab to Excel
  • fixed the AV error caused by switching to the "SERP Snippet" tab with no active project selected
  • fixed the incorrect accounting of the inbound and outbound links and their mapping in different blocks of the program
  • fixed the incorrect mapping of the percent indicators in the "Level" node
  • fixed the incorrect mapping of the "Content-Type" node data in the project scan statistics
  • fixed the unstable data filtering for an exact match
  • fixed the problem of continuing the paused site scans
  • fixed the incorrect HTTP/S sign when exporting reports to Excel
  • fixed the incorrect accounting of canonical URLs during the website scanning
  • fixed the incorrect accounting of Allow and Disallow statistics for robots.txt
  • fixed the incorrect accounting of images ALT and TITLE attributes
  • fixed the unstable PageRank calculation
  • corrected the typo "Alernate" to "Alternate" on the Info tab
More info: SiteAnalyzer 2.1
  •  


majentoTopic starter

We have released a new version of SiteAnalyzer, we'd love to hear feedback and suggestions for the new version.

New version SiteAnalyzer 2.2









What's new:

  • we added a custom filters module to search for content on websites during crawling
  • we added internal links chart of a website
  • when dragging a node of a graph, its child elements are dragged together with it
  • when clicking a node of a graph, inbound and outbound links have different colors
  • the visualization graph legend is interactive now (when clicking the elements in the legend, the corresponding nodes are highlighted on the graph)
  • X-Robots Tag option added to the website crawling settings
  • optimized parsing for H1-H6 headers that utilize classes
  • software hang in the final stage of large projects crawling is eliminated
  • fixed incorrect statistics representation of duplicate meta descriptions
  • fixed incorrect statistics representation of 404 pages
  • blocked URLs in Robots.txt now return status code 600
  • Response Time parameter is calculated more accurately now
  • fixed incorrect Sitemap.xml generation
  • redirects are displayed more accurate now
  • sorting by URLs is more accurate now
More info: SiteAnalyzer 2.2
  •