Pages:
Author

Topic: **Updated** Bitcoin Search Engine: moving into final release stage. (Read 1006 times)

newbie
Activity: 28
Merit: 0
Great idea but you are not the first:

http://www.bitcoinsphere.com/

http://bitcoinmagazine.com/the-bitcoin-search-engine-launches/

There was another one around a while ago (forgot the name) but it seems to have faded into obscurity.

However, I really like some of your ideas such as a crypto "better business bureau"  Wink

Ethicoin, when it is released, will function as a crypto charity (amongst many other things)

In fact - your project is eligible for funding.

PM me if you would like to be added to the Ethicoin receiver files upon release (we will have a thread for this soon in the alt coin forum)

hero member
Activity: 686
Merit: 504
always the student, never the master.
r3wt,
Have you seen these guys?

http://www.80legs.com/


just a little bit outta my price range broski... i'm no where near ready to deploy yet anyway. i'm still making some changes to the code. edit: besides, i already have my my crawler or as i call it, "the spider" setup how i want it. im trying to make a firm plan of action how to deploy it in a large scale. i was thinking of setting up external access to my db, and creating nodes on as many free webhosting accounts with some little dinky front website to run my crawler on. this way i could deploy quite a few spiders at once. the key is finding a webhost who will let me connect to the database externally and from multiple ip addresses simultaneously. then i was thinking about building a shell script that would move seamlessly from one crawl to the next, so that i can manage my nodes efficiently

they have a free plan
doesn't do what i need.
sr. member
Activity: 392
Merit: 250
♫ A wave came crashing like a fist to the jaw ♫
r3wt,
Have you seen these guys?

http://www.80legs.com/


just a little bit outta my price range broski... i'm no where near ready to deploy yet anyway. i'm still making some changes to the code. edit: besides, i already have my my crawler or as i call it, "the spider" setup how i want it. im trying to make a firm plan of action how to deploy it in a large scale. i was thinking of setting up external access to my db, and creating nodes on as many free webhosting accounts with some little dinky front website to run my crawler on. this way i could deploy quite a few spiders at once. the key is finding a webhost who will let me connect to the database externally and from multiple ip addresses simultaneously. then i was thinking about building a shell script that would move seamlessly from one crawl to the next, so that i can manage my nodes efficiently

they have a free plan
sr. member
Activity: 392
Merit: 250
♫ A wave came crashing like a fist to the jaw ♫
Interesting idea, but I think it would be hard to cover all bitcoin related sites. You'd still need google for complete coverage I'm afraid.
you don't seem to understand how this works, so ill explain it for you. google works as a spider. every searchengine does. it searches domains for links and tags. it indexes these tags to and matches them to a link list. when you search google, you pull up a list of links that closely match your search term. basically if i set my search engine to index to a depth of six, and all bitcoins sites had some random hash tag like "hashBTC1321ljwljaslsdjaflasorewralesfjlsadfjasdl99999999999" i could set the spider(search engine) to leave the domain, and it would find every website on the internet with that tag in its meta tag list.

Each site can use there own address,

Code:
hero member
Activity: 686
Merit: 504
always the student, never the master.
r3wt,
Have you seen these guys?

http://www.80legs.com/


just a little bit outta my price range broski... i'm no where near ready to deploy yet anyway. i'm still making some changes to the code. edit: besides, i already have my my crawler or as i call it, "the spider" setup how i want it. im trying to make a firm plan of action how to deploy it in a large scale. i was thinking of setting up external access to my db, and creating nodes on as many free webhosting accounts with some little dinky front website to run my crawler on. this way i could deploy quite a few spiders at once. the key is finding a webhost who will let me connect to the database externally and from multiple ip addresses simultaneously. then i was thinking about building a shell script that would move seamlessly from one crawl to the next, so that i can manage my nodes efficiently
sr. member
Activity: 392
Merit: 250
♫ A wave came crashing like a fist to the jaw ♫
hero member
Activity: 686
Merit: 504
always the student, never the master.
Interesting idea, but I think it would be hard to cover all bitcoin related sites. You'd still need google for complete coverage I'm afraid.
you don't seem to understand how this works, so ill explain it for you. google works as a spider. every searchengine does. it searches domains for links and tags. it indexes these tags to and matches them to a link list. when you search google, you pull up a list of links that closely match your search term. basically if i set my search engine to index to a depth of six, and all bitcoins sites had some random hash tag like "hashBTC1321ljwljaslsdjaflasorewralesfjlsadfjasdl99999999999" i could set the spider(search engine) to leave the domain, and it would find every website on the internet with that tag in its meta tag list.

This would take months. the spider would just be cranking band with crawling the internet. the solution? what google does. "web master tools" is a really fancy multi featured way of letting web site owners do the part of the heavy lifting of the search engine process. this is the next step in my development. right now, i've got a form that gets your email address a description of your website, the url of your homepage and makes you fill out a captcha. when you hit submit if it verifies, it adds your site to qeue to be indexed by the spider. for my idea to be possible, on a global scale, i would need a virtual private server with very liberal mysql connection allowances. the trick is to run multiple instances of the spider at once to crawl pages and speed up the time it takes index. then you have temp tables which serve to hold site information until the spider crawls it. you have to clear these manually. So in essence, you can compete with google, but you have to have a damn nice hosting setup with a very flexible memory and mysql database access abilities. my ideal setup would be 100 simultaneous database connections, unlimited bandwidth, 3 databases, 10gb diskspace.
newbie
Activity: 22
Merit: 0
Interesting idea, but I think it would be hard to cover all bitcoin related sites. You'd still need google for complete coverage I'm afraid.
sr. member
Activity: 392
Merit: 250
♫ A wave came crashing like a fist to the jaw ♫
We should propose a btc meta tag, the purpose of which is to easily define a website as about btc rathar than having to parse it from the content on the site itself.
hero member
Activity: 686
Merit: 504
always the student, never the master.
here you go. this is what a search engine looks like

http://coinbit.pw/search/search.php
hero member
Activity: 686
Merit: 504
always the student, never the master.
Hi everyone.
Glad you did this poll...

Have you read this? Posted on the main page of Bitcoin Magazine
http://bitcoinmagazine.com/the-bitcoin-search-engine-launches/

BTC

no offense, but thats just a slapped together hodge podge of code

here, i'll upload my searchengine engine to a webhost so you can see what a real search engine looks like... stand by.
newbie
Activity: 19
Merit: 0
That would be nice Smiley  I know there's a couple of lists of bitcoin sites, however there are probably hundreds or thousands more (such as myself) who have a site that's not on that list.

What do you suppose a good tag to put in my header would be to be found on such a search engine? (or even google at the moment)
newbie
Activity: 4
Merit: 0
Hi everyone.
Glad you did this poll...

Have you read this? Posted on the main page of Bitcoin Magazine
http://bitcoinmagazine.com/the-bitcoin-search-engine-launches/

BTC
full member
Activity: 224
Merit: 100
One bitcoin to rule them all!
I voted "possibly" as it would depend on how it's implemented etc.

Interested, but not too far from indifferent.
newbie
Activity: 6
Merit: 0
You mean a much better alternative to bitcoinwatch.com? Yes! I would like to use some more sophisticated "Bitcoin dashboard". Possibility to search in gathered data isn't bad either.
member
Activity: 118
Merit: 10
full member
Activity: 238
Merit: 100
The Bitcoin Catalog ---> Get Started!
Definitely interested!
newbie
Activity: 32
Merit: 0
I'm interested to use this.
member
Activity: 116
Merit: 10
Curious to see how you execute this.  the easier it is to access information on bitcoin and how/where to use, the better IMO.
member
Activity: 111
Merit: 10
“Everyone Is A Bank”
I can offer help - Ruby on Rails development
and provide a free hosting for that Smiley
Pages:
Jump to: