Commit 811c559f authored by Michael Murtaugh's avatar Michael Murtaugh

old timeline description from cc.py

parent 4d148802
---
title: Collage Code
date: September 2018
---
Collage code is a way of editing media files using a lightweight linking using [media fragments](https://www.w3.org/TR/media-frags/). It's inspired by experiments with making *wiki-style* systems based on earlier experiments with SRT and lightweight markup such as textile (as well as markdown).
CC works with "link bundles" ... basically links made in the same paragraph that are considered as a single pool, effectively gluing together different pieces like fragments of collage on a pasteboard. For instance, the links:
[A#t=10]
[B]
If B is time based, this is equal to:
A#t=10 <--> B#t=0
This is a *collage link*. It says that B (from time 0) should be attached to A at time 10 seconds.
![](images/timeline01.png)
Links as transformations
----------------------------
Collage links can be seen as translations that map coordinates in one frame of reference to another. As such, the transformation can take place in two directions. Many links can be represented simply by a translation, but some links can also stretch or scale values.
Moving from A to B:
Adjust both start & end += (B.start - A.start)
and from B to A:
Adjust both start & end += (A.start - B.start)
Links with all four start & end values specified might also involve stretching such as:
E#t=10,20 <--> F#01:00,01:30
In this case the transformation involves both a shift in position, as well as a stretch (x3) which in temporal terms means a reduction in playback speed to 1/3 of normal (or a speed up going from F to E).
TODO: formula for translation with stretching.
Link intersection / compaction
-------------------------------------
Given the links:
[A#t=10]
[B]
Consider adding a similar link between B and C.
[B#t=10]
[C]
![](images/timeline02.png)
Now C is indirectly related to A. It would be convenient to express the relationship directly in terms of A and C:
A#t=20 <--> C
How does this relationship get calculated. Consider the two links:
A#t=10 <--> B
B#t=10 <--> C
![](images/linkintersection.png)
They are joined by the common resource B. These two links can be reduced to a single link by *link intersection*:
* Find the intersection of the *inner* or shared fragments.
* Transform this intersection back to the outer timelines via their respective links.
The intervals of B & B#t=10 is (0, end) and (10, end). The intersection is then (10, end).
Transforming this via the first link results in A#t=20, and via the second as C#t=0 or just C, so the final link is:
A#t=20 <--> C
Flattening a timeline
-----------------------
Cascading timelines can be usefully used to group different resources to place on independent timelines, then use collage links to mix the different sources. Link intersections can be used to rewrite an entire collection of (indirectly related) elements to become relative to a single timeline (and time range if desired).
A#t=10 <--> B
B#t=10 <--> C
C#t=30 <--> audio.ogg
![](images/timeline03.png)
To flatten a link, l, consider all overlaps of l. Intersect the links, push the new links onto a stack to also consider (recurse). To generalize the process for any specific fragment, consider an automatically generated link that links from some target timeline T to the requested Fragment. So to flatten A, start with the link T -- A:
T -- A
+ A#t=10 -- B
= T#t=10 -- B
+ B#t=10 -- C
= T#t=20 -- C
+ C#t=30 -- audio.ogg
= T#t=50 -- audio.ogg
So the final flattened timeline:
T#t=10 -- B
T#t=20 -- C
T#t=50 -- audio.ogg
And for instance flatten(A#30,60):
T -- A#30,60
* A#t=10 -- B
( Intersection is A#30,60 )
= T -- B#20,50
* B#t=10 -- C
( Intersection is B#20,50 )
= T -- C#10,40
* C#t=30 -- audio.ogg
( Intersection is C#30,40 )
= T#20,30 -- audio.ogg#t=0,10
or:
T -- B#20,50
T -- C#10,40
T#20,30 -- audio.ogg#t=0,10
![](images/timeline04.png)
In preparing say a commandline editor command, these results would then be filtered to only involve actual resources (audio.ogg) and exclude the intermediary timelines.
Playlist / Tracks
-----------------------
Once links are flattened (and thus all relative to a single timeline), it may be convenient to render the links to the simpler "playlist" or "tracks" representation. The *layout_tracks* function does this.
The method is as follows:
* Compute the "interesting times" (sorted union of start and end times of all the "inner" fragments)
* For each interval, for each overlapping interval i, split the interval at the interval ending into i1, i2. In the case that i1 ends at the interval ending, then i2 will be null. Place i1 on the current stack, joining / extending an existing element if possible. Insert i2 (if not null) to the start of list of remaining intervals to be placed (to give it priority when placing to ensure join/extend where possible).
* As necessary, add tracks. When adding a track, add a blank element of duration equal to the start of the current interval.
* After placing all overlapping tracks, pad any remaining (unused) tracks with blank elements (joining/extending if possible).
Remaining issues
--------------------
* Conditional to limit non-overlapping intervals
* Interval completion (resolving links 3 points-4) ... when does this happen (in flattening?!)
* Media consulation for durations... using this data in rendering tracks
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment