FreeVPS Post Count Script.
#1
NOT AVAILABLE ANYMORE!



Well, after some debugging in Conan's topic here where he tried to modify Transfusion's post count script. He asked me to see if I can adapt it to another myBB forum (which I won't mention here). But rather than converting the original script I ended up re-implementing a new one that uses only PHP functionality and no external class or libraries like the original one did. Transfusion's script uses a PHP library called PHP Simple HTML DOM Parser or something like that which makes it quite annoying to debug the existing code. Probably the reason Conan couldn't convert it to another forum.

The code is well documented and should be fairly easy for anyone wishing to adapt it to other forums or to their needs. The script should be quite efficient because the parsing isn't done in PHP code but rather by the C PHP extension libXML which powers the DOM manipulation in PHP. The script uses cURL to fetch the web pages and DOMDocument to transform the HTML data into a DOM tree then XPath to navigate the DOM structure.

The library extracts the exact time stamp from the post date which (in theory) should make it very accurate. Transfusion's script only checks the month and ignore's the year which could lead to inaccurate results if posts match with the months of a year before the current one.

The post searching is quite strict and if the script doesn't find what it needs for a post when it parses it then it ignores the post. Also the script exits on the slightest missing data to avoid freezing in some loop or fetching pages without stopping. Performs type checking for important data to be sure nothing is wrong and should be fairly safe compared to Transfusion's script.

Code:
The code that used to be here was moved to the Github repository: https://github.com/iSLC/FreeVPS

I've only tested the code on my local WAMP stack and worked perfectly. You shouldn't bother to test it on shared hosts (especially the free ones) because it probably wont work. Most shared hosts disable outside access from scripts for security reasons. The script wasn't tested to much but (in theory) should work out of the box. (just make sure your PHP installation has cURL extension enabled)

I still haven't converted the script the the forum Conan asked me to but I'm guessing the script is well documented for anyone to modify. I just didn't had the time to do it and I don't use that forum which is why the script was tested only on FreeVPS. Even though the forum he specified is also myBB it's very different then the myBB version on FreeVPS so the script won't work without some changes.

NOTE: Some forum's are hidden from guest users (like Help & Support) and since the script is considered a guest when crawling, it won't see those posts to count them. (A workaround for this limitation was implemented as of version 0.7) The posts from ignored forums (like Introductions and SPAM/Testing) will be skipped automatically from the count.

Screenshot of the page for users that own a VPS:
Spoiler: show
[Image: Ashampoo_Snap_20140424_22h00m58s_001__zps1d1b4f44.png]
Screenshot of the page for users that don't own a VPS:
Spoiler: show
[Image: Ashampoo_Snap_20140424_22h15m37s_001__zps226cd746.png]
Screenshot of the page for the banned or unapproved users:
Spoiler: show
[Image: Ashampoo_Snap_20140424_22h16m09s_002__zpsd5e1f924.png]

The script can be downloaded from the Github repository here. A temporary preview of the script is available at the following address:
Code:
The address was removed because the IP of the VPS was banned from the forum! Please resort to using your own VPS or a temporary WAMP, XAMP, LAMP, MAMP etc. stack.
ATTENTION: You can also use one of the mirrors bellow. These mirrors are provided by users and I do not guarantee their availability.:
Spoiler: show

ATTENTION: I won't host the script fro too long because the script is made to be self hosted and you are free to host it on your own. If however you are able to host it and you think that your host is more permanent then please PM me the link and I'll add it here for anyone who needs it.

ATTENTION: Currently the script is quite bugged because I haven't got the time to test it. Some of the code should look like copy > paste just to get the job done. That's because I tried to finish the script before I had to take care of some things in real life.

ATTENTION: Please report any issues you have here so that I can fix them or to tell you why they're happening.

Changelog:
Spoiler: show
  • 0.7
    • Implemented a temporary workaround for the guest issue to make the post count more accurate.
    • Added the option to count how many times the script was accessed.
    • A few other fixes and add-ons  that are not worth listing here. (I also don't remember most of them)
  • 0.6
    • Added the Bootstrap interface for a better preview of the extracted details.
    • Fixed many of the issues concerning the different HTML structure between the user groups.
    • Improved the user profile crawling code to get more useful fields and fixed a few other issues.
    • A few other fixes and add-ons  that are not worth listing here. (I also don't remember most of them)
  • 0.5
    • Fixed some issues with banned or unapproved users.
    • Improved the user profile crawling code to get more useful fields.
    • Improved the post search crawling code to also get a list of the counted posts.
    • A few other fixes and add-ons  that are not worth listing here. (I also don't remember most of them)
  • 0.4
    • Prevent the script from exiting when querying for username elements on banned users.
  • 0.3
    • Prevent the script from querying for username elements on a Missing-User page which would cause the script to exit.
  • 0.2
    • Prevent the script from querying for post elements on a No-Results page which would cause the script to exit.
  • 0.1
    • Initial release.
#2
Only 2 more things Smile
Code:
FAILED: The DOM query for the user post elements failed to retrieve a valid element.
// only dmm's account works
// http://REMOVED2623/script.php?userid=1
// http://REMOVED2623/script.php?userid=2

and can i copy part of transfusions script that makes it print to a picture instead of a text? is it compatible
Note: I already asked permission from Transfusion to edit his code Smile
#3
i am getting the same error when using ther peoples user id
REMOVED PER REQUEST
error:
Code:
FAILED: The DOM query for the user post elements failed to retrieve a valid element.

EDIT:
when i post another posts it would give the error
i have 52 posts then when i posts another it dont work anymore
#4
Well, I fixed most of the issues but the script won't count the posts from forums hidden to Guests users. Forums like "Help & Support" are not available to guest users and the script is considered a Guest user when crawling Cheese
#5
Then why not give it an username like for google crawler?
So Say We All
#6
(2014-04-17, 8:09:36 pm)xaoc Wrote:  Then why not give it an username like for google crawler?

I would not ask the administrators of this forum to do such thing. I wouldn't want to be held responsible for anything that would create possibilities to harm this forum later Cheese
#7
It wont cause problems if it's a simple user. And you can state : Use this plugin at your own risk. Cheese
So Say We All
#8
Or you could setup a 'bot' account to crawl the posts which are hidden for Guests, quite simple to do with post data & cURL.
I've done this for several other sites etc
#9
(2014-04-17, 8:36:21 pm)xaoc Wrote:  It wont cause problems if it's a simple user. And you can state : Use this plugin at your own risk. Cheese

I know it won't do any harm but just in case, I don't want to have fingers pointed at me. And also this isn't a plugin. It's just a PHP crawler script that downloads pages like your browser does and counts the posts it finds.

(2014-04-17, 8:47:35 pm)nomis002 Wrote:  Or you could setup a 'bot' account to crawl the posts which are hidden for Guests, quite simple to do with post data & cURL.
I've done this for several other sites etc

I was actually planing to improve it in a later version to actually bypass the Guest barrier. And to add a few other features that would improve the accuracy. I'll start working as soon as I get some good sleep because I'm tired as hell right now Cheese
#10
Would you please explain me how to install and use it? That would be helpful.
"FIRE" available for free download. Click here to listen to it.




Users browsing this thread: 1 Guest(s)