Background
Virgil Griffith a Disruptive Technologist at the California Institute of Technology made me interested in one aspect of data mining on the internet. The search for sources of the information we find on the internet. This crucial bit of source reference which we as scientists all time through our careers try to find out, who wrote about a certain topic, or has it been written about before.
Internet is kind of a basket containing all types of information and everything can be found within it. It does not discriminate about its originality or how genuine the information is, it only discriminate on its popularity on the internet through its search engines.
Virgil Griffith made a program called WikiScanner, to be found at the internet as http://wikiscanner.virgil.gr/.
For those of you not knowing Virgil Griffith, I will give a short story on him here. I may be wrong and leave out things that should be here, but it is only meant as a short intro to the person, and nothing else.
Virgil Griffith (born 1983 in Alabama USA), also known as Romanpoet, is a hacker, known for his involvement with a 2003 lawsuit with the Blackboard Inc. company and his creation of the WikiScanner. He has also published papers on artificial life. He is a visiting Researcher at the Santa Fe Institute and at the present a graduate student in the Computation and Neural Systems department at Caltech.
Qualification of internet information
Performing a qualification of all information on internet can be a challenge to say it at the least. Someone would call it impossible to perform a check on the information and who is the originator of this information.
To just see where Wikipedia gets its information from, you can use the WikiScanner and get an idea at least. Some surprises may pop up as you use the tool, and some results maybe confirm your already established idea of where information comes from.
Maybe this gives you a guideline of where information on the internet comes from, I do not know.
WikiScanner results
Griffith found that 34.5 million contributors where anonymous edits, which is around 21% of Wikipedia content contributors in total.
There are in excess of 2,668,095 different organizations in database.
He found 187,529 different orgs with at least 1 edit.
There were some other interesting findings such as;
There is a different % of anonymous edits by country
The CIA does in fact edit Wikipedia
FOIA lawsuit filed over Mike Huckabee white-washing
Dutch princess white-washes connections to drug baron.
Politicians and corporations do in fact hire staff to police their pages.
Follow-Ups
If you are interested in doing data mining for yourself, why not try wikiscanner and in addition use some of these toolboxes available for you.
The General Architecture for Text Engineering called GATE; http://gate.ac.uk/
MySQL / Python/ Ruby
Text Similarity tools; http://freshmeat.net/projects/levenshtein/
Data-mining tools; http://wiki2.issuecrawler.net/twiki/bin/view/Dmi/DmiTools
Got a Question? Ask.
Ask the community a question about this article:
Frequently Asked Questions
Navman f series
By: robin | 19-08-2008
has anybody gotan instuctionmanual
HoPXw can i convert the source code into assembly ...
By: amy | 19-08-2008
hoPXw can i convert the source code into assembly code in java ?
Help with Face Book
By: Linda | 18-08-2008
How in the world do I find an American version of Face Book?? When I click on their link to join....I get a page in French...every time....I cannot read it!!
Transformers For Sale
By: Parts guy | 18-08-2008
I have a 3 phase transformer that has HV 600, LV208. Can I connect 3 phase wiring to the LV208 and get HV600 for machinery use? In effect, reversing the operation of the transformer. Parts Guy
Wine & travel
By: Richard Engelsman | 18-08-2008
travling from canada to the US , how much wine can we bring Back?
were from the US visting canada and the wine is for our personal use.
2sp mtr changing gears without shifting
By: chuck | 17-08-2008
2sp mtr changing gears without shifting
Q&A Powered by:
More from Stig Kristoffersen
Iphone on Fire!
By: Stig Kristoffersen | 19/09/2008 | Cell Phones
Events of meltdowns and fires have been reported by consumers in various parts of the world regarding the new iPhone.
Events have been reported where the connection between the iPhone and the charger cord melts due to high temperatures. There have been reported events where the battery in the new iPod has taken fire as well.
The Key to Prosper and Have Wealth in Your Life
By: Stig Kristoffersen | 13/09/2008 | Coaching
Many books and articles cover this topic, as well as many web sites which offer you the secrets for only a few USD or Euros. Just join their communities and you are closer to the salvation.
Let us start with this question, whenever did you hear about these 4 things?
Knowledge is power
Time is Money
Be a realist
Law of Attraction
Let me tell you something about these 4 topics and let you in on a secret. You have the power, time and realism to attract wealth to you and your family.
Making a New Start in Life…
By: Stig Kristoffersen | 10/09/2008 | Coaching
If you look for a new opportunity, you have two choices, either find new work or start up for yourself.
You should be looking for new work rather than a new job. If you are thinking of starting up your own business or consulting activity, you should wonder were to start look.
You have to be clear about who you are, what your skills are or what your competencies are, and last but not least where you want to go. In addition you need to define a transition timeline with appropriate
The Key to Unlocking Your Power to Wealth Ands Success Today !
By: Stig Kristoffersen | 09/09/2008 | Coaching
We present for you the solution to achieve your success and wealth today. A simple 7-step formula guided by 15 "laws" of attraction will guide you through the process. If you follow these steps and "laws", you will as several other wealthy individuals before you, succeed and prosper in the future.
It is up to your self, go the path and commit yourself to the new life, and then you are on your way to prosperity.
You are to be the Next Millionaire!
By: Stig Kristoffersen | 08/09/2008 | Coaching
This article will give you some insight that will enoucrage you to reach your goals set out in your life.
Whether you want to become a millionaire, or you just want to be happy with your family or to live in harmony with the people around you, here is the solution presented to you. It is simple, it is genious and very effective. But be careful what you wish for, it will come through.
Let Me Tell You How You Can Become a Millionaire
By: Stig Kristoffersen | 08/09/2008 | Career Management
The Key to Unlocking Your Power to
Wealth and Success Today!
This article will give you some insight in how to succeed reaching your dreams and acheive the success you need.
Anyone have the ability, only few knows how to unlock this ability into action and to go through with it. Commitment to the theory bhind the methodology and the willingness to go through with a plan to acheive it, will make it come through. Just read more and you will reach the goals you set out for yourself.
Google Gets Competition From Cuil
By: Stig Kristoffersen | 28/07/2008 | SEO
Previous employees of Google are amongst the entrepreneurs of the new search engine called Cuil.
Bigger and better than Google.com?
Vanco Exploration and Production Rights - Still Under Discussions - Case is Now in the International Court of Arbitration in Stockholm
By: Stig Kristoffersen | 11/07/2008 | Politics
US' Vanco delays filing lawsuit in Ukraine over canceled PSA.
The case is now in the International Court of Arbitration in Stockholm, and Vanco is preparing a lawsuit against the Ukrainian government to retain their right for exploration in the offshore Black Sea. The production sharing Agreement made is deemed not valid by the Ukrainian authorities after some irregularities were discovered regarding its award as well as some questions were risen towards the joint venture group lead by Vanco.
Technology 
About the Author:
