Deduplicating apparent duplicates?

In an effort to use the blog to document ongoing SouthComb activities, this entry is the first in a long line (I hope) of ongoing conversations within the development team. This project is designed to provide transparent access to scholarly materials related to the U.S. South. Some of the issues in providing transparent access are more easily understood. However, I have a feeling this one will be a bit tricky.

Building a search application means running the risk of collecting what might appear as duplicate records. Web sites, for example, can have multiple pages (home page, navigation page, contact page, site map page, multiple content pages). To an automated web crawler, the content of these pages may be different, but the title is the same. This causes a good bit of confusion when trying to find specific content.

So how do we develop SouthComb so that each record it provides is a unique and meaningful record ?

How are scholars using SouthComb?

How scholars use SouthComb is on the forefront of my mind. We are talking with faculty on our Advisory Board about how they would use SouthComb in the classroom. Hearing that its use is obvious, I seek a clearer definition. Faculty recommend SouthComb as a scholarly resource for research papers. Faculty also think that the news is the most striking feature and one people will notice. We have also heard good feedback that SouthComb is a good reference for Centers and Programs.

SouthComb is designed to be a transparent and scholarly search of materials related to the U.S. South. More specifically, SouthComb is scholarly in the sense that it uses what we call a "white list" of web sources, digital archive OAI providers, and newspapers to seed the searches. The website does not harvest or crawl information that we do not know of. All of these soures have been reviewed by a team of scholars and new sources suggested will also undergo the same rigourous process. The list of all of these sources is found on the website at the top of the naviagational links in the left, hence our process is transparent.

So how do you use SouthComb?

Southerns Spaces Call for Proposals Due Jan 30, 2009

PROPOSALS DUE JANUARY 30, 2009
Southern Spaces invites proposals for its second series, "Documentary Expressions and the American South." Submissions should engage with documentary work in and around the U.S. South by combining innovative commentary about documentary expressions with digital media elements like images, maps, audio, or video. Southern Spaces also welcomes submissions of riginal documentary projects.

400 to 600-word proposals should include:

• a description of the major ideas, arguments, and sources for the project
• the estimated page length of project text
• a list of multimedia (photo, audio, video) components
• a brief statement about why this work is a strong fit for this series

Please visit Southern Spaces for more information. http://www.southernspaces.org/abo_cfp_doc.html

Search the news!

SouthComb collects news stories about the South from over 250 newspapers from states considered part of the U.S. South. Search for news by keyword, by region or by subject category. Remember, we provide results for both the individual RSS news feed AND the newspaper source itself.

SouthComb on Facebook

Show SouthComb your support and become a fan on Facebook. Check SouthComb's Facebook page to see new additions to our collection, new features on the website or announcements from across the community.

SouthComb beta release

We are pleased to release www.southcomb.org to our Southern Studies colleagues who have watched SouthComb develop over the last year. We encourage you to use SouthComb any way you can, in class, out of class, for research, or for fun! We will be adding new features to the web site so please look for announcements sent via email or online!