Author

Topic: SigBuddy - Now open sourced (Read 700 times)

hero member
Activity: 699
Merit: 501
June 20, 2016, 01:49:52 PM
#8
Finally ,you took my suggestion to to make it open to Public ,nice.I shall fork the repository soon.I'm thinking if stuff can be replaced with PHP ? We can do much url validations and keeping users data using SQL.I have never really used Jsoup for parsing ,I have to learn that.

It was a really good advice, I didn't realize how important it was, until the project was almost killed.

For Jsoup, I'm sure you'll be really good at it. Its exactly like JQuery (When your selecting items by ID, Class, Attr).
legendary
Activity: 1988
Merit: 1317
Get your game girl
June 20, 2016, 01:34:39 PM
#7
Finally ,you took my suggestion to to make it open to Public ,nice.I shall fork the repository soon.I'm thinking if stuff can be replaced with PHP ? We can do much url validations and keeping users data using SQL.I have never really used Jsoup for parsing ,I have to learn that.
hero member
Activity: 699
Merit: 501
June 20, 2016, 09:05:57 AM
#6
The BBcode issue is this.

In the HTMLDocument package that I've created, the Loader class requires special HTTPS header, took about 2 days to get the correct ones.
Without "application/x-www-form-urlencoded" the posted data will not be accepted, so will always fail, this is also the same for the user-agent.
If you look at your userprofile information page (to change signatures) the submit button has a charset of "ISO-8859-1" I've replicated everything & even tried combinations of UTF-8 & ISO-8859-1 in the HTTPS post header, but each time, symbols like ticks, in the signature will be broken.

I will reproduce an example and leave it on my signature for the next 2 days. So you can analyse it.


The BBcode encoding issue is the only problem that's stopping the project.

Ok, I'll take a look. Have you tried using JSoup's connection stuff to handle sending the data instead of using HttpsURLConnection?

Can you also put the BBCode for your sig here (in code brackets)?

Server BBCode character encoding problem

Target result
Quote

Server result
Quote
staff
Activity: 3458
Merit: 6793
Just writing some code
June 20, 2016, 08:48:30 AM
#5
The BBcode issue is this.

In the HTMLDocument package that I've created, the Loader class requires special HTTPS header, took about 2 days to get the correct ones.
Without "application/x-www-form-urlencoded" the posted data will not be accepted, so will always fail, this is also the same for the user-agent.
If you look at your userprofile information page (to change signatures) the submit button has a charset of "ISO-8859-1" I've replicated everything & even tried combinations of UTF-8 & ISO-8859-1 in the HTTPS post header, but each time, symbols like ticks, in the signature will be broken.

I will reproduce an example and leave it on my signature for the next 2 days. So you can analyse it.


The BBcode encoding issue is the only problem that's stopping the project.

Ok, I'll take a look. Have you tried using JSoup's connection stuff to handle sending the data instead of using HttpsURLConnection?

Can you also put the BBCode for your sig here (in code brackets)?
hero member
Activity: 699
Merit: 501
June 20, 2016, 08:39:03 AM
#4
I might be able to take on this project.

Where is the source code? It isn't the repo you linked.

What exactly is the BBcode issue you were having?

Sorry i had to delete and update the project. It contained some cookies in there.

The BBcode issue is this.

In the HTMLDocument package that I've created, the Loader class requires special HTTPS header, took about 2 days to get the correct ones.
Without "application/x-www-form-urlencoded" the posted data will not be accepted, so will always fail, this is also the same for the user-agent.
If you look at your userprofile information page (to change signatures) the submit button has a charset of "ISO-8859-1" I've replicated everything & even tried combinations of UTF-8 & ISO-8859-1 in the HTTPS post header, but each time, symbols like ticks, in the signature will be broken.

I will reproduce an example and leave it on my signature for the next 2 days. So you can analyse it.


The BBcode encoding issue is the only problem that's stopping the project.

Interesting, so you developed this to help people running signature campaigns or is this for something else? I often wondered how people run the signature campaigns and how to track what they pay, etc. I guess you developed this tool to help track it?

Yup it pretty much does everything. Hopefully someone else might continue this project, complete all the abstract commands, and connect all the dots together to make this intelligent system a reality.



--
--

I truly don't think that this project will be released/exist without the developers on this thread. You guys are special, and some of you have been tracking, informing & guiding this project since it started. The keys are on your hands, and I know this small community can be entrusted to make into something great.
staff
Activity: 3458
Merit: 6793
Just writing some code
June 20, 2016, 08:24:09 AM
#3
I might be able to take on this project.

Where is the source code? It isn't the repo you linked.

What exactly is the BBcode issue you were having?
legendary
Activity: 1512
Merit: 1057
SpacePirate.io
June 20, 2016, 08:17:26 AM
#2
Interesting, so you developed this to help people running signature campaigns or is this for something else? I often wondered how people run the signature campaigns and how to track what they pay, etc. I guess you developed this tool to help track it?
hero member
Activity: 699
Merit: 501
June 20, 2016, 08:07:47 AM
#1
Project Threads




Current features

--> Detect signatures & personal texts
--> Post signatures & personal texts
--> Has RPC capabilities
--> Comes with chrome extension


Notes: I said that i wouldn't be releasing this project to the public. Things have changed. I believe someone else could continue or find this source code very helpful. [Full disclosure] I found that most of my time would have been spent finding a solution to the broken BBcode HTTPS encoding issue, because of this, I've decided to drop this project.

Development: Jsoup is used to scrape parse HTTP texts. You can read more about the library online. For each connections, the first connection will fail (in an attempt to take control of the session), the second connection will pass (where you can change signatures). If the users refreshes their browser after the first failed connection, then the second connection will also fail.

Before you can use the program, you must replace the userCookie with the user cookie, example:

setUserCookie("PHPSESSID=doijsodijogsijgi849gijksdgsdo; SMFCookie129=osppsdkg%foidjfobj%kfjkdfjljdf%;");


From my tests, this value is almost, always constant....

You must also set the users SC token. This can be found on the logout button on bitcointalk:

setUserSC("ksdfosaofi923982h39fj23gj3gkl");


The purpose of the browser extension, was to collect these information from the users browser, send it to our Database, where the Java program/server can access it and use them to save signatures....



If you would like my information/documentation on how to build/use this program, please contact me.


What to do

In the ExtractToUser class, I have only provided a simple function which detects whether the profileInformation edit was successful. If it's successful, what you need to do now, is create another method which will go to the users profile, and scrape & update all the information in the DB. you also need to scrape the signature which will now be displayed in a HTML format. This will allow you to compare it (any time) to check if the user has altered or removed it.


Running the Server

The default location of the server is at: http://localhost:8000/?query=profile

You can choose how you want to handle HTTP connection from the PHP server in the HTTPhandle pakcage, Request Class, on its "handle" method


Design Work Flow


The extensions should collect the document.cookie & sc token, send it to the PHP server every 30 minutes. The PHP server will take the data, give it to the Java App server, (which will try to save the profileInfo - without any changes) to confirm whether it's valid or not. If it's valid, it will update the database record.

Using Jsoup, every hour the Java will go through each campaign, and all the users which are in those campaigns. Since it saved their HTML signature from when it set it, it will compare whatever it find now, to what is in the database. It will do the same thing for the users personal texts. It will go to the users profile, and look at posts. It will collect posts, from the date the campaign started, to the present. Each post has date & time on its header, it will help you keep track... Each post also has a href link, which contains a unique ID, for example: "https://bitcointalksearch.org/topic/sigbuddy-now-open-sourced-1520107" this value 1520107.msg15292702, or the whole link, should be hashd in sha1 to be used as a unique id for each post. These posts will be collected & their contents. You must remove the quotes from each post.

What is considered spam?

I compared the average posts of several members & texts (100 lines), and the minimum amount of effort, to not be considered as a spam is having a post with more than 9 words, and 42 characters. That is it.

Once each post has been collected and confirmed not as spam, they should be scraped and sent off to your PHP server, which will insert it into your database, so the ad campaign owner can review and select which posts they will not pay for.

How are ads setup?

The campaign manager should be allowed to select upto 5 main boards where posts will automatically be rejected. They should also select minimum ranks, whether the user has negative trust or not & any other feature which you like.

What about extensions?

Some users might wish not to be included in campaigns. Because of this, a freeze button was introduced. When a user clicks it, the server will not place them in any campaigns until they have unfrozen their accounts. Users who are also inactive for more than 4 days, are put in a frozen state.

The active status in the extension is used to show whether a user is currently in a campaign, or whether the user has been banned. A user has been banned, when they receive too much kicks from advertisers. Their accounts should also be reviewed before its banned from using this service.


Jump to: