How do I get a Wyam pipeline of documents based of a comma-delimited meta value from a previous pipeline? - wyam

Question

I have a Wyam pipeline called "Posts" filled with documents. Some of these documents have a Tags meta value, which is a comma-delimited list of tags. For example, let's say it has three documents, with Tags meta of:
gumby,pokey
gumby,oscar
oscar,kermit
I want a new pipeline filled with one document for each unique tag found in all documents in the "Posts" pipeline. These documents should have the tag in a meta value called TagName.
So, the above values should result in a new pipeline consisting of four documents, with the TagName meta values of:
gumby
pokey
oscar
kermit
Here is my solution. This technically works, but I feel like it's inefficient, and I'm pretty sure there has to be a better way.
Documents(c => c.Documents["Posts"]
.Select(d => d.String("Tags", string.Empty))
.SelectMany(s => s.Split(",".ToCharArray()))
.Select(s => s.Trim().ToLower())
.Distinct()
.Select(s => c.GetNewDocument(
string.Empty,
new List<KeyValuePair<string, object>>()
{
new KeyValuePair<string, object>("TagName", s)
}
))
)
So, I'm calling Documents and passing in a ContextConfig which:
Gets the documents from "Posts" (I have a collection of documents)
Selects the Tags meta value (now I have a collection of strings)
Splits this on the comma (a bigger collection of strings)
then trims and lower cases (still a collection of strings)
De-dupes it (a smaller collection of strings)
Then creates a new document for each value in the list, with am empty body and a TagName value for the string (I should end up with a collection of new documents)
Again, this works. But is there a better way?

Answer 1

That's actually not bad at all - part of the challenge here is getting the comma-separated list of tags into something that can be processed by a LINQ expression or similar. That part is probably unavoidable and accounts for 3 of the lines in your expression.
That said, Wyam does provide a little help here with the ToLookup() extension (see the bottom of this page: http://wyam.io/getting-started/concepts).
Here's how that might look (this code is from a self-contained LINQPad script and would need to be adjusted for use in a Wyam config file):
public void Main()
{
Engine engine = new Engine();
engine.Pipelines.Add("Posts",
new PostsDocuments(),
new Meta("TagArray", (doc, ctx) => doc.String("Tags")
.ToLowerInvariant().Split(',').Select(x => x.Trim()).ToArray())
);
engine.Pipelines.Add("Tags",
new Documents(ctx => ctx.Documents["Posts"]
.ToLookup<string>("TagArray")
.Select(x => ctx.GetNewDocument(new MetadataItems { { "TagName", x.Key } }))),
new Execute((doc, ctx) =>
{
Console.WriteLine(doc["TagName"]);
return null;
})
);
engine.Execute();
}
public class PostsDocuments : IModule
{
public IEnumerable<IDocument> Execute(IReadOnlyList<IDocument> inputs, IExecutionContext context)
{
yield return context.GetNewDocument(new MetadataItems { { "Tags", "gumby,pokey" } });
yield return context.GetNewDocument(new MetadataItems { { "Tags", "gumby,oscar" } });
yield return context.GetNewDocument(new MetadataItems { { "Tags", "oscar,kermit" } });
}
}
Output:
gumby
pokey
oscar
kermit
A lot of that is just housekeeping to set up the fake environment for testing. The important part that you're looking for is this:
engine.Pipelines.Add("Tags",
new Documents(ctx => ctx.Documents["Posts"]
.ToLookup<string>("TagArray")
.Select(x => ctx.GetNewDocument(new MetadataItems { { "TagName", x.Key } }))),
// ...
);
Note that we still have to do the work of getting the comma delimited tags list into an array - it's just happening earlier up in the "Posts" pipeline.