Stakeholders in Using the Accounting

Introduction

Creating a curriculum accounting is a non-trivial amount of work.

Optimally, it should be the starting point of a curriculum. In my experience, however, it rarely is. Its lack is generally discovered as an afterthought when operational challenges and scale reveal gaps. These challenges and gaps mount up like debt (similar to developers’ “technical debt”), demanding more and more operational resources until they stifle execution speed. Invariably, this will occur at a critical and inconvenient time. Then the accounting is noticed missing, lamented as such, and must be built (under duress) to pay down the debt that’s been rung up.

Why ought we make that initial outlay of time to create an accounting and maintain it while we build out the curriculum? Specifically, to whose benefit is it to have this accounting? Those are the subjects for this section.

The Promise

Ultimately our stakeholders all have one quality in common:

They demand our assurance that we have been thoughtful, accurate, and careful in defining the map to the educational territory.

Ultimately the promise is fulfilled by the vibrant maintenance of the accounting. Let’s see how our accounting and our stakeholders interact.

Example

In the following, assume an “Introduction to JavaScript” module‘s accounting. Suppose it teaches Learning Goals sets JS1, JS2, JS3 and JS4, JS5, JS6 through two lessons “Lesson 1” and “Lesson 2.”

Internal Stakeholders

In an organization that delivers training at scale, the systems that document the Learning Goals and their containing macro-objects must be likewise scalable, explorable and self-documenting. The presence of an accounting means that we can easily scale the organization by bringing on new staff, new offices, etc. with minimal marginal overhead. The accounting teaches humans how to teach it so that more humans can engage with it in a virtuous cycle. The accounting enables us to facilitate communication and pin down “hot spots” that require further refinement.

Let’s try to identify workflows for certain internal stakeholder archetypes.

On-Site Educators

Accounting Feature: Checksum

For On-site Educators, the accounting provides a means for debugging and tracing their students’ learning journeys, as we’ve already covered. Another benefit, more closely related to promise fulfillment is that the accounting provides verification that we teach what we said we’d teach through a “checksum” operation.

Suppose:

  1. We link the module to those Learning Goals
  2. We also link Learning Goals to Modules through Lessons

That means through two pathways we can get from the module to its Learning Goals. We can use this fact to create a verifying “checksum.” If the module’s Learning Goal set matches the Learning Goal set through the lessons, we can signal “OK.” If the “checksum” verification fails, the system should notify the curators of the accounting.

Accounting Feature: Precision Feedback

The accounting also helps on-the-ground implementers provide profoundly useful feedback. Consider the precision of the following statement and what its alternative might be:

“I see in Introduction to JavaScript we set goal JS1 and it’s exercised in Lesson 2 of the module, but I don’t think the requirement is accurately exercised by test #2 in indexText.js as written.”

Everyone starts on the same page in terms of knowing the intention of the materials and can participate in the maintenance easily.

Accounting Feature: Precision Assessment / Assessment Review Feedback

We just saw how the accounting helps on a per-Lesson basis, but at the module level, it makes writing assessments easier. No question can be asked unless it points to a Learning Goal. If there aren’t enough questions, we need to make more Learning Goals (or make the existing goals finer). Adding more Learning Goals in a lesson requires them to be bonded to the module (given our checksum logic, above).

Consider a population of Questions which are each members of an Assessment which itself is bonded to a Module.

Each Question points to a Learning Goal as well as a Rubric Prompt.

Thus, by running some sort of a algorithm:

  • all the questions can be packaged as an assessment
  • a common rubric for evaluating the assessment can be emitted
  • a directed graph graphic can be generated that shows the tree of Learning Goals

As the accounting evolves, the assessments and their rubrics evolve as well, for minimal marginal input.

The accounting obviously has benefits for internal, teaching-oriented collaborators. But it also has benefits for internal, unknown collaborators.

Unknown Internal Collaborators

One of the biggest measures of success for a technology is the number of individuals who use it in surprising ways. It was a short “microposting” feature of the Odeo streaming platform that became Twitter. It was the clever bendings of the rules by users that made HyperCard the paradigm-shifting application it became. This accounting does not pretend to belong to such great company, but we’ve found that other peer organizations that might have only the faintest of tangential associations with curriculum also benefit from the accounting.

Admissions Counselor

Let’s consider the case of an admissions counselor, a role neither in the direct- nor dotted-line reporting structure of most academic teams. This role is fulfilled by someone who is helping a qualified lead decide whether training with us is the right decision.

If an admissions counselor is trying to answer technical questions beyond their depth, a meta-data field on the module can provide key sales points per a given module.

Or consider the case where a learner has seen some new acronym on a job posting and wants to know whether it’s covered in the program. The admissions counselor can search for the Learning Goal and find the lesson or module where the Learning Goal is present (and even trace the Learning Goal to the assessment where its mastery is verified!).

Compliance Department

As a final stretch for this system, consider a compliance department. If the accounting is inaccessible, the curriculum development team must do spreadsheet / document preparation tasks, meetings, etc. in order to provide the paperwork required for an audit. If always-up, always-fresh, always-accurate reports can be constructed as a “view” of the accounting, compliance can help itself.

External Collaborators

Most external collaborators have an interest in using an accounting to resolve questions that pertain to their particular activity. We’ll cover these only in a cursory fashion as these concerns generally overlap with the cases described above.

Learners

A noteworthy external consumer is our learner, our student.

The learner can debug their own learning process by being able to work out signals from assessments, difficulties in Lessons, and discussions with their coaches. The aforementioned assistance to communication provided by the accounting is transferable to the learner/coach context.

A notable case is the perennial “Why am I learning this?” question. For students whose work occurs after- or before-work, it’s easy to lose context of what their present work was designed to do. With access to the accounting they can find their current Lesson see how it fits to a Module and see how a Module is a prerequisite for an outcome. Knowing this, they know whether they ought push on a little bit longer to close out a module or if it’s something they need to tackle with a fresh set of eyes in the morning.

Hiring Market for Graduates

A graduate learner or employee partnerships manager (another internal role) should be able to discuss the accounting during interviews so that would-be employers can fairly assess where any misalignments or further training might be required. Where differences are identified, those can be delivered as feedback to curriculum development or they can be identified as points of growth for the newly-hired.

Licensing Bodies

In order to obtain state licensing as a school, candidate-schools must be able to submit a compendium of what is taught to the licensing bodies. The fully expanded set of atomic Learning Goals should be easily calcuable and easily shared. With sufficiently sophisticated views, a goal state could be pursued: one view that renders all the compliance documentation required at the push of a button — or better yet, which is always up-to-date without any buttons being pushed!

Conclusion

Having identified and discussed a wide variety of stakeholders, a certain set of criteria has begun to emerge as to what we want from an accounting system:

  1. Can identify Learning Goals and bundle them into macro-objects and those macro-objects into nth level abstractions
  2. Cycles are possible and permissible
  3. Calculated attributes are possible. Examples:
  • Two numerical attributes can be summed
  • Two attributes can have textual operations (join, delimited by a character)
  • Attributes can be calculated using a limited set of programmatic expressions
  1. Schema changes can be made quickly and possibly even discussed as “changesets”
  2. Schema meta-documentation is preserved in an accessible fashion
  3. Accessibility to many levels of stakeholders
  4. Low requirements of information technology education in order to:
  • Update schema
  • Add content
  • Display content in discipline-appropriate views e.g. “report views” might be appropriate for sales staff to access and print; “rubric views” might be appropriate for on-site educators to use as a checklist in progress discussions with learners

Next: Solutions for Modeling the Accounting