The Human Relations Area Files announces the retirement of Douglas Avison Black, IT Team Leader and Software Application Developer. After an incredible tenure spanning over three decades, Doug will be greatly missed by all. His contributions have shaped, and will continue to shape, the development of HRAF as an organization well into the future.

Doug has witnessed many HRAF milestones, including the transition from paper files and microfiche to wholly digital collections, first on CD-ROM and later on the web. His role has involved establishing document structures, production processes, and conversion specifications underlying the eHRAF databases. He has been the lead database engineer, as well as maintaining both the Outline of World Cultures (OWC) and Outline of Cultural Materials (OCM), the classification systems at the foundation of our indexed collections.

Hundreds of member institutions and countless users of our databases have benefited from the tireless dedication of Douglas Black at the helm of our IT department. The unparalleled human-indexed rendering of full-text data at the center of the eHRAF World Cultures and eHRAF Archaeology databases allows us to offer a unique focus on the paragraph level of ethnographic texts. Doug’s efforts over the years have been key to producing this powerful, comparative research tool which has become indispensable for generations of anthropologists and cultural researchers. It is with great pride that we celebrate his three decades at HRAF.

Early Years

Doug’s exemplary service to HRAF began when he joined the organization as a secretary in 1985. At this time, the organization had no digital presence whatsoever since the Internet had not yet been invented. It was while working on a giant word processing machine in the lobby of the HRAF building in New Haven, Connecticut, that Doug’s interest in learning more about programming was sparked. It was also around this time that Doug met his future wife, M. Marlene Martin.

Marlene started her HRAF career as an analyst in the mid-70s before becoming the Supervising Analyst and Bibliographer, and eventually a Specialist in Electronic Publications. She applied her considerable technical expertise to the development, production, and maintenance of eHRAF and other electronic publications and projects. Doug and Marlene began dating shortly after Doug joined HRAF. They shared a lifelong passion for many things, including an obsession with the computer language SGML, which was undoubtedly nurtured throughout their mutual employment at HRAF.

Perhaps serendipitously, Doug’s first years at HRAF coincided with a turning point in the organization’s history. Established in 1949, HRAF was known for its paper file collections housed at universities around the world in imposing, floor-to-ceiling metal filing cabinets. Many longtime fans of the Paper Files still recall them fondly, although few complete collections remain today. Wider distribution of the HRAF Collection of Ethnography was facilitated in 1958 with the development of the HRAF Microfiche Files. However, by the early 1980s, revenues from the microfiche were in decline, and the future of the non-profit organization was uncertain.

At this critical point in HRAF history, the decision before the organization was whether or not take the substantial risk of going fully electronic. In the 1980s, this meant digitizing HRAF’s document collections onto CD-ROM. Such a bold move would indeed be costly, but if done successfully, could position the organization well for the rapid expansion of future digital products and services that could only be hinted at in the years prior to the promise of a World Wide Web.

Going Digital

Fiscally responsible decision-makers at HRAF sought to remain cautious. However, another school of thought was also represented: Melvin Ember, then Chairman of the Board, held an opposing point of view. He championed the movement towards digitization for HRAF as early as 1986. Unsurprisingly, Doug’s future wife, Marlene Martin, shared this view. She took on a technical lead in making it possible for HRAF to embrace going fully electronic.

Around this time, Doug began taking computing classes at Columbia University with a growing interest in programming. In addition to a transition away from paper and microfiche files, a second organizational decision was taken to shift to using a generalized markup language for programmatically describing document structure. This would provide the foundation for effective electronic publication on CD and, later, to the expansion of eHRAF onto the web.

The first electronic product released was a series of CDs produced with DynaText, an SGML-based publishing tool introduced in 1990. SGML (standard generalized markup language) would become the predecessor to both HTML and XML, all of which are markup languages used to display, structure, store, and transfer data and information on the Internet.

Marlene first discovered SGML as a new way of marking up document structures in order to format text for electronic use. Most CDs up until to that time which contained text documents or publications employed a proprietary markup language specific to that particular brand of CD or publisher. SGML was an attempt to generalize the markup so that it could be used across companies and products. Tim Berners-Lee later used SGML as a model to create HTML for the web.

Ahead of Its Time

Thanks in part to the long-term ambitions and thoughtful consideration of document structure by Marlene, Doug, Melvin Ember, and others, HRAF was ahead of its time on the path towards digitization. Internal discussion had begun as early as 1986 and the process was underway by the late 1980s to early 1990s. HRAF quickly found itself at the forefront of new innovations in electronic publishing. It also put into stark relief how distinct the HRAF Files were from other data collections since their inception.

Doug reflected on the dynamic in the office at this time:

Back then, we – and especially Melvin – were actually thinking about artificial intelligence: machine analyzing a text document for a topic, to supplement human indexing. That caused a bit of a dispute between Marlene and Mel. She was a big believer in human indexing because she started out as an analyst and believed – probably rightly so – that human indexing is superior to machine indexing. But it’s amazing that we were already thinking about machine indexing back then. Mel had an information scientist friend that was prominent in machine learning, so he had additional information about it from there.

He hastens to add that HRAF is still ensnared at the center of these same technological movements today. For example, the ongoing NSF-funded iKLEWS project seeks to enhance indexing and eHRAF though machine learning by leveraging natural language processing. Doug (and Marlene’s) early steps at taking HRAF digital, as well as continued efforts producing and refining the eHRAF databases for the web, formed the critical foundation upon which iKLEWS builds today.

HRAF’s Cross-Cultural CDs were launched in 1993. Shortly after, the first online version of the eHRAF Collection of Ethnography was made available in 1997. Hosted by the Digital Library Production Service at the University of Michigan, it became one of the earliest full-text electronic databases to exist on the web. At the time, most e-databases were comprised of bibliographic records only. It was followed by eHRAF Archaeology in 1999.

Doug consulted for HRAF from 1995 to 2005. During this time, he worked on SGML and document structure, as well as building the first version of the eHRAF World Cultures application in-house at HRAF to be served from Yale University, as it is today. It went live on the web in 2008.

The HRAF Files, from paper to digital, have always been much more than just a collection of documents. At the core of its programming, eHRAF breaks documents down to the paragraph level. Our analysts then process each and every paragraph of over 880,000 pages by adding subject codes to categorize the data. This human step makes the data more meaningful for searching complex concepts across large data sets, such as when conducting comparative cross-cultural research. To display on the web, eHRAF must then reassemble the individual paragraphs back into a familiar document from, so that researchers can read their selections in the original context if desired. One could argue that in eHRAF, the document as a whole has become greater than the sum of its parts.

It may surprise some readers to discover that a small, non-profit anthropological organization has so often been at the forefront of technological advancement. Doug imparts a positive lesson that he learned from this experience:

I’ve been here long enough to see that HRAF is very, very often – to me – at the cutting edge of issues in information processing, and science and development. But it’s often hard to keep up, with the small capital that we have. It’s hard to run a software development firm with just a couple of developers, or at times even just one developer. In so many areas, it’s just hard to do that. The positive is, we always seem to be where it’s all happening in terms of what we’re trying to do, and to some extent, succeeding.

Legacy for the Future

Once Doug had taken on development of eHRAF in-house at Yale in the early 2000s, he and co-developer Matthew Roth worked to improve the front end with more user-friendly revisions, creating the familiar interface that users know and love today. More recently, as Doug approached his retirement, Matt Roth incorporated lessons learned from their shared experience of developing the first in-house eHRAF database into the latest redesign of eHRAF World Cultures. The most recent iteration – formally launching in 2022 – makes substantial updates to both the front and back end of the application.

Doug has worked to strengthen HRAF’s basis for growth as we welcome the next generation of eHRAF application. His unified long-term vision for the organization has guided his leadership within and beyond IT. Doug’s keen understanding of document structure, categories, and relationships between and across data – also labors of love for his late wife Marlene – remain omnipresent in the everyday functionality of eHRAF. His indefatigable search for sensible solutions to questions without clear answers will be remembered by all as a guiding tenet for future IT meetings.

How did Doug find the motivation to stay at HRAF for so many years?

I’ve managed to do it so long, I think, because it’s never been finished. There’s always been problems and bugs, and so forth … There’s also always been some debates. I used to tell people, if you just agree that I’m right, then I’ll leave! It’s hard to walk away, because you know it’s not done yet.

If you have ever used eHRAF for cross-cultural research or to find ethnographic or archaeological data, Doug has been a key architect behind your success, as he is behind many of the achievements that HRAF has enjoyed to date. Upon his retirement, the HRAF Board of Directors will recognize Doug’s extraordinary contributions to HRAF and his outstanding legacy with a commemorative plaque to be showcased in our building at 755 Prospect Street in New Haven.

We hope that you will join us in congratulating Doug Black and thanking him for his service to HRAF, anthropology, and information sciences. It is with a heavy heart that we wish him all the very best for the future and a well-deserved, happy, and healthy retirement.

September 29, 2021