Collect & Create and Analyze & Collaborate slices from the larger biomedical data lifecycle wheelWhere to Start

When it comes to managing data and documents within a workgroup, you must implement processes that work best for your team. Even if you are in the middle of a project, a good place to start is to define directory structures and naming conventions. Defining structure early in the process provides an organizational foundation for all your data and document files.

Best practices for setting up your projects prior to data collection and creation include:

  • Establishing Naming Conventions for your files and using them consistently will ensure maximum access to your data and records

  • Building Directory Structures refers to the method of classifying and organizing data sets to make them more useful

  • Using a system for active project Version Control can help keep track of all sorts of files, including text documents and analysis code. And avoid Final_v2_rev3_final_FINAL.docx headaches!

  • Creating plain text or README Files that contain information about other files in a folder for each distinct dataset

Scenario Examples

  • File evolution and migration

    Read about how to manage growing and changing projects

    As the complexity of the project and data evolves, you may find a need to update conventions set-up at the start of the project. For example, when a new project is spawned from a long multi-year project that has different needs or you inherit data and files transferred from another institution that used different naming conventions than your institution. If there is already a structure in place, see what fits for your team, and modify as needed. If your project already has hundreds of unmanaged files or in a structure that does not work for your team, you'll need to weigh the cost and time to implement the above strategies. 

    There are three main options to tackle this issue, from least to most time intensive to implement:

    1. Only new files will fully follow the team's collect & create policies (old files will remain in old system)
    2. Pull forward old files from the old system into the new as they are used
    3. All files will be moved into the team's system

    Using README files, when inheriting project files or getting a handle on unmanaged files, will help document choices made not only for the team, but for yourself when you need to review several months or years later.

  • Organizing space for accessibility, security, and team roles

    Read about how to manage collaborative projects

    Having a set of policies, at the outset will go a long way to manage the growth of information. For example, how to format dates in file names, e.g., use a prefix format YYYY-MMDD, or create  readme file once the directory structure is set. Each of these will be different for different teams. There may also be variations based on shared team folders vs individual member folders.

    Additional issues to consider in the team workspace include accessibility, manipulability, security, and team roles. The goal of the team space is to be able to have efficient access to the thousands of files created over time so that the task of finding what is needed when it is needed will not create undue stress and will increase productivity for the group. For shared workspaces, it is important to have more than one owner assigned, so as personal move on and off the project, continuity of access to the files and role assignments can be maintained. Owners are able to set security permissions for various individuals and subgroups, monitoring how the group is adhering to the agreed upon conventions, and institute changes as business practices change or when the present structure seems to be failing to meet business needs. 

    Some of the security or permissions issues may have to reside within the I.T. department, but will vary depending on the internal structure of the group’s information system hardware. For example, if you are using Microsoft Teams, much of the access and security is managed within the team, where as if you are using a University server, security assignments will reside with the compliance office or IT/DevOps teams.  

    Though, much of what you need to manage is in electronic format, there are usually some paper records that also need to be maintained. For example, a signed data use agreement, which may be scanned in as a PDF.  If there is a paper-based system of record, you should consider how the two systems need to dovetail together. You may also need to consider University retention schedules for important documents, which support the business functions and activities of the team.