Commit c116b8f2 authored by Alexandre Leray's avatar Alexandre Leray
Browse files

Initial commit

aa.wiki was previously merged with aa.core. This is the result of the
seperation of the core with the wiki part, aiming to make it easier to
build on top of aa.core and the many modules made for active archives.
parents
*.pyc
# db files, sqlite & RDF hashes
*.db
*.bak
*~
.svn
*.swp
*.orig
venv/
repositories/
run/media/
This diff is collapsed.
# How can I embed an RSS feed?
Create a link to the feed with an embed relation, like this:
[[ embed::http://url/of/the/rss/feed.rss ]]
# How can I put an image in the background?
Because Active Archives wiki syntax is based on Markdown, it is possible to use
plain HTML in your documents, including css directives. If you want to put an
image in the background of your page you can put somewhere in the source of the
page the following snippet:
<style>
body {
background: url('http://repo.oralsite.stdin.fr/Jerome%20Bel/Jerome%20Bel_show_wrk_01.jpg');
-moz-background-size: auto 100%; /* Firefox 3.6 */
background-size: auto 100%;
background-repeat: no-repeat;
background-position:center;
}
</style>
For more information about the styling properties and their options, you can
refer to <http://www.w3schools.com/cssref/default.asp>
FIXME: The instruction in this file are outdated since the refactoring of
aacore.
This is the INSTALL file for Active Archives (AA). This document provides
convenient steps so a user/developer can have AA running quickly.
Dependencies
============
Please first install these software in order to get AA running properly:
- Django += 1.3 <http://www.djangoproject.org/>
- SQLite3 <http://www.sqlite.org/>
- PySQLite2 <http://trac.edgewall.org/wiki/PySqlite>
- Git <http://git-scm.com/>
- Git-Python <http://gitorious.org/git-python>
- Google-diff-match-patch <http://code.google.com/p/google-diff-match-patch/>
- Python html5lib <http://code.google.com/p/html5lib/>
- Python markdown += 2.1 <http://www.freewisdom.org/projects/python-markdown/>
- Python dateutil <http://labix.org/python-dateutil>
- Pygraphviz <http://networkx.lanl.gov/pygraphviz/>
- Python librdf (Redland) <http://librdf.org/>
On Ubuntu, this should do the trick:
sudo apt-get install python-pip
sudo pip install django
sudo pip install gitpython
sudo pip install html5lib
sudo pip install markdown
sudo pip install diff-match-patch
sudo pip install python-dateutil
sudo pip install pygraphviz
sudo apt-get install python-librdf
Additionally, you'll need the following for the RDF sniffers:
- Exiftool
- Ffmpeg
- Imagemagick
Quick Installation
==================
1. Clone the repository onto your machine
git clone git@git.constantvzw.org:aa.core.git
1. Once you've installed the required dependencies:
cd /path/to/aa.core/run
2. Build the database:
python manage syncdb
The prompt will ask for the admin infos and fixtures will be loaded.
3. Run the django webserver:
python manage.py runserver
4. Configure the project domain name at:
<http://localhost:8000/admin/sites/site/1/>
Typically the value must be "localhost:8000" if you are running the project
on a local server.
vim: ft=markdown:
The Active Archives Video Wiki inverts the paradigm of uploading resources into
a centralized server and instead allows resources to remain "active", in-place
and online. Caching and proxy functionality allow (light-weight) copies of
resources to be manipulated and preserved even as the original sources change
or become (temporarily) unavailable.
Strategically, the project aims to clarify some of the "cloudy" aspects of Web
2.0 regarding issues of licensing, sharing, ownership, access, and longevity of
online material. Designed to break open the "black box" of online video, users
are encouraged to write with video, creating new compositions made from
collages of disparate (online) elements.
This is alpha software with known bugs. It runs, and works at least some of the
time, but use at your own risk.
For more information about the project, see <http://activearchives.org/>
High priority
=============
- Skip tag
- Create new page: default content that give clues
- First page directly created?
- Give a location to the first annotation
- add annotation button: predefined location + meaningful text too.
* Improve history
* Be able to revert a page
- layer palette doesn't refresh
Low priority
============
- BUG: negative positions
- bug with column ":" in url: sould be escaped by the wikify function.
- shift to enable the grid snapping
- javascript scrollbars (less ugly)
- draggable help
- extra menu to show/hide annotation by criterias
- Metadata in the sidebar
- edit locks?
- button to remove the about attribute/ drag the @ to remove the about attribute + confirmation
- "attach" boxes
- with data-anchor, data-position (north, south, east, west etc.)
- configure serve to serve best html5 video/audio
- RSS feeds
- Still a problem with non consecutive header levels...
- list pages filter ?
Ideas
=====
- html5 audio/video: preload="auto"?
- How can RDF be useful for collective commenting and annotating?
- rdf metadata to create relation between pages. EG. translation.
- move page
- How to preserve the title attribute with the new syntax ([[embed::ressource]])?
- manual sectioning "8<---"? What about the <br /> tag which is semantically correct for this?
- Notion of focus for media, to know which one is being controlled
- different types of landmarks... what syntax?
Oralsite workshop remarks
=========================
- video leads text/text leads video
- embed webpages = iframe?
- A nice sniffer: `file`:
➜ ~ file /tmp/marqueurs.txt
/tmp/marqueurs.txt: ASCII English text, with very long lines
- annotation negative position happens when one rescales the annotation "outside" the screen.
- manual to update
- how to change the embed wrapper?
Meeting 01/2012
===============
- drop-down css don't work on time-based medias
Navigation
----------
- how to come back to TOC when sidebar is closed
- TOC links weird
- Bug when resizing a box it goes up
- home page different than publication homepage
- cinematic mode shows hidden boxes in edit mode
- drag and drop classes
- more explicit class names
- show sound files in the timeline
- background appears in cinematic mode?
- remove decorations with classes
- play/read mode: choose to put a border or not on the boxes
- if timeline, mode name = play, otherwise mode name = read.
- possibility to keep titles
- play/pause/skip to next landmark
- use the timeline to fly to a box (already working)
Bug
---
- carriage return on titles
- history: date sections not accurate
- Jonathan Burrows on Scores: play mode: sidebar opened the images overflow
- lightbox: missing navigation buttons
- renumber section when one gets deleted
- Question marks in wiki links are mistakenly decoded
[[What's the Score Now?]] --> http://localhost:8000/pages/What%27s_the_Score_Now/?/edit/
- Pages names must not contain non-ascii characters
Meeting 27/01/2012
==================
- Namespace for Ritz
LGRU
====
- sidebar opened by default.
- editable stylesheets
- file upload
- Rester en contact avec Bachir pour l'hébergement + donner accès à Active Archives. Serveur des laboratoires.
from django.contrib import admin
from models import Page
class PageAdmin(admin.ModelAdmin):
list_display = ("name", "content")
search_fields = ("name", "content")
admin.site.register(Page, PageAdmin)
# This file is part of Active Archives.
# Copyright 2006-2011 the Active Archives contributors (see AUTHORS)
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Also add information on how to contact you by electronic and paper mail.
import re
from aawiki.mdx.mdx_sectionedit import (TIMECODE_HEADER, spliterator)
from aawiki.timecode import (timecode_fromsecs, timecode_tosecs)
def audacity_to_srt(data, explicit=False):
"""
>>> data = '''90,610022 90,610022 first section
... 345,271874 345,271874 second section'''
>>> print(audacity_to_srt(data).strip())
00:01:30.610 -->
<BLANKLINE>
first section
<BLANKLINE>
00:05:45.272 -->
<BLANKLINE>
second section
>>> data = '''90,610022 345,271874 first section
... 345,271874 512,573912 second section'''
>>> print(audacity_to_srt(data).strip())
00:01:30.610 -->
<BLANKLINE>
first section
<BLANKLINE>
00:05:45.272 --> 00:08:32.574
<BLANKLINE>
second section
>>> data = '''90,610022 345,271874 first section
... 345,271874 512,573912 second section'''
>>> print(audacity_to_srt(data, explicit=True).strip())
00:01:30.610 --> 00:05:45.272
<BLANKLINE>
first section
<BLANKLINE>
00:05:45.272 --> 00:08:32.574
<BLANKLINE>
second section
>>> data = '''90,610022 345,271874 first section
... 345,271874 512,573912'''
>>> print(audacity_to_srt(data).strip())
00:01:30.610 --> 00:05:45.272
<BLANKLINE>
first section
<BLANKLINE>
00:05:45.272 --> 00:08:32.574
<BLANKLINE>
second section
"""
stack = []
for line in data.splitlines():
try:
(start, end, body) = tuple(line.split(None, 2))
except ValueError:
try:
# A marker without label
(start, end) = tuple(line.split(None, 1))
body = ""
except ValueError:
# A blank line? Get lost!
break
start = float(start.replace(',', '.'))
end = float(end.replace(',', '.'))
start = timecode_fromsecs(start, alwaysfract=True,
alwayshours=True, fractdelim=',')
end = timecode_fromsecs(end, alwaysfract=True,
alwayshours=True, fractdelim=',')
# If the end time equals the start time we ommit it.
if end == start:
end = ""
if not explicit:
# Deletes previous end time if equal to actual start time
if len(stack) and stack[-1]['end'] == start:
stack[-1]['end'] = ""
body = body.replace(r'\n', '\n')
stack.append({'start': start, 'end': end, 'body': body})
template = "{e[start]} --> {e[end]}\n\n{e[body]}\n\n"
return "".join([template.format(e=e) for e in stack])
def srt_to_audacity(data, force_endtime=False):
"""docstring for srt_to_audacity"""
# FIXME: UnicodeDecodeError...
pattern = re.compile(TIMECODE_HEADER, re.I | re.M | re.X)
stack = []
for t in spliterator(pattern, data, returnLeading=0):
m = pattern.match(t[0]).groupdict()
if force_endtime:
if len(stack) and stack[-1]['end'] == '':
stack[-1]['end'] = timecode_tosecs(m['start'])
end = timecode_tosecs(m['end']) or ''
else:
end = timecode_tosecs(m['end']) or timecode_tosecs(m['start'])
stack.append({
'start': timecode_tosecs(m['start']),
'end': end,
'body': t[1].strip('\n'),
})
template = "{e[start]}\t{e[end]}\t{e[body]}\n"
return "".join([template.format(e=e) for e in stack])
if __name__ == "__main__":
import doctest
doctest.testmod()
f = open('/tmp/bla.srt', 'r')
data = f.read()
f.close()
#print(audacity_to_srt(data))
print(srt_to_audacity(data))
# This file is part of Active Archives.
# Copyright 2006-2012 the Active Archives contributors (see AUTHORS)
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
# Also add information on how to contact you by electronic and paper mail.
import urlparse
import urllib2
import html5lib
import lxml
import re
from aacore import rdfutils
import subprocess
import os.path
import feedparser
from utils import get_rdf_model
from textwrap import dedent
class AAFilter(object):
def __init__(self, arguments, stdin):
self.arguments = arguments or ""
self.parsed_arguments = {}
self.stdin = stdin
self.stdout = stdin.copy()
self.stdout['local_path'] = self.get_next_path()
self.stdout['local_url'] = self.get_next_url()
if self.validate():
return self.run()
def validate(self):
return True
@staticmethod
def uri_to_path(uri):
return urlparse.urlparse(uri).path
def get_next_path(self):
extension = os.path.splitext(self.stdin['local_path'])[1]
return "%s|%s:%s%s" % (self.stdin['local_path'], self.__class__.__name__,
self.arguments, extension)
def get_next_url(self):
extension = os.path.splitext(self.stdin['local_url'])[1]
return "%s|%s:%s%s" % (self.stdin['local_url'], self.__class__.__name__,
self.arguments, extension)
class AAFilterEmbed(AAFilter):
name = "embed"
def run(self):
embed_style = None
if self.arguments:
embed_style = self.arguments
else:
q = dedent("""\
PREFIX dc:<http://purl.org/dc/elements/1.1/>
PREFIX aa:<http://activearchives.org/terms/>
PREFIX http:<http://www.w3.org/Protocols/rfc2616/>
PREFIX media:<http://search.yahoo.com/mrss/>
SELECT ?ctype ?format ?audiocodec ?videocodec
WHERE {{
OPTIONAL {{ <%(URL)s> http:content_type ?ctype . }}
OPTIONAL {{ <%(URL)s> dc:format ?format . }}
OPTIONAL {{ <%(URL)s> media:audio_codec ?audiocodec . }}
OPTIONAL {{ <%(URL)s> media:video_codec ?videocodec . }}
}}""".strip() % {'URL': self.stdin['original_url']})
model = get_rdf_model()
b = {}
for row in rdfutils.query(q, model):
for name in row:
b[name] = rdfutils.rdfnode(row.get(name))
break
# TODO: move to templates
if b.get('ctype') in ("image/jpeg", "image/png", "image/gif"):
embed_style = "img"
elif b.get('ctype') in ("video/ogg", "video/webm") or (b.get('videocodec') in ("theora", "vp8")):
embed_style = "html5video"
elif b.get('ctype') in ("audio/ogg", ) or (b.get('audiocodec') == "vorbis" and (not b.get('videocodec'))):
embed_style = "html5audio"
elif b.get('ctype') in ("text/html", ):
embed_style = "iframe"
elif b.get('ctype') in ("application/rss+xml", "text/xml", "application/atom+xml"):
embed_style = "feed"
else:
embed_style = None
# TODO: move to templates
if embed_style == "img":
self.stdout['output'] = '<img src="%s" />' % self.stdin['local_url']
elif embed_style == "html5video":
#self.stdout['output'] = '<video class="player" controls src="%s" />' % self.stdin['local_url']
# Temporarily fixes the local serveur serving ogg with the wronf mimetype
self.stdout['output'] = '<video class="player" controls src="%s" />' % self.stdin['original_url']
elif embed_style == "html5audio":
#self.stdout['output'] = '<audio class="player" controls src="%s" />' % self.stdin['local_url']
# Temporarily fixes the local serveur serving ogg with the wronf mimetype
self.stdout['output'] = '<audio class="player" controls src="%s" />' % self.stdin['original_url']
elif embed_style == "iframe":
self.stdout['output'] = '<iframe src="%s"></iframe>' % self.stdin['local_url']
elif embed_style == "feed":
feed = feedparser.parse(self.stdin['local_url'])
self.stdout['output'] = u''
for entry in feed['entries'][:4]:
self.stdout['output'] += u'<div>'
self.stdout['output'] += u'<h3><a href="%s">%s</a></h3>' % (entry.link, entry.title)
self.stdout['output'] += u'<div>'
self.stdout['output'] += entry.summary
self.stdout['output'] += u'</div>'
self.stdout['output'] += u'</div>'
else:
self.stdout['output'] = "<p>Unable to detect embed type</p>"
class AAFilterXPath(AAFilter):
""" Takes a url as input value and an xpath as argument.
Returns a collection of html elements
usage:
{{ "http://fr.wikipedia.org/wiki/Antonio_Ferrara"|xpath:"//h2" }}
"""
name = "xpath"
#def validate(self):
#sizepat = re.compile(r"(?P<width>\d+)px", re.I)
#match = sizepat.match(self.arguments)
#if match:
#self.parsed_arguments['width'] = match.groupdict()['width']
#return True
def absolutize_refs (self, baseurl, lxmlnode):
for elt in lxml.cssselect.CSSSelector("*[src]")(lxmlnode):
elt.set('src', urlparse.urljoin(baseurl, elt.get("src")))
return lxmlnode
def run(self):
request = urllib2.Request(self.stdin['original_url'])
request.add_header("User-Agent", "Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.1.5) Gecko/20091109 Ubuntu/9.10 (karmic) Firefox/3.5.5")
stdin = urllib2.urlopen(request)
htmlparser = html5lib.HTMLParser(tree=html5lib.treebuilders.getTreeBuilder("lxml"),
namespaceHTMLElements=False)
page = htmlparser.parse(stdin)
p = page.xpath(self.arguments)
if p:
self.stdout['output'] = "\n".join([lxml.etree.tostring(self.absolutize_refs(self.stdin['original_url'], item)) for item in p])
#self.stdout['output'] = "\n".join([lxml.etree.tostring(self.absolutize_refs(self.stdin['original_url'], item), encoding='utf-8') for item in p])
else:
self.stdout['output'] = "<p>Your query for %s did not return any elements</p>" % self.arguments
class AAFilterBW(AAFilter):
name = "bw"
def run(self):
if not os.path.exists(self.stdout['local_path']):
cmd = 'convert -colorspace gray %s %s' % (self.stdin['local_path'],
self.stdout['local_path'])
p1 = subprocess.Popen(cmd.split(" "), stdout=subprocess.PIPE,
stdin=subprocess.PIPE)
(stdout_data, stderr_data) = p1.communicate()
#(stdout_data, stderr_data) = p1.communicate(input=self.stdin)
self.stdout['output'] = "Conversion successful. Use the embed filter to display your ressource in the page"
class AAFilterResize(AAFilter):
name = "resize"
def validate(self):
sizepat = re.compile(r"(?P<width>\d+)px", re.I)
match = sizepat.match(self.arguments)
if match:
self.parsed_arguments['width'] = match.groupdict()['width']
return True
def run(self):
if not os.path.exists(self.stdout['local_path']):
cmd = 'convert -resize %s %s %s' % (self.parsed_arguments['width'],
self.stdin['local_path'],
self.stdout['local_path'])
p1 = subprocess.Popen(cmd.split(" "), stdout=subprocess.PIPE,
stdin=subprocess.PIPE)
#(stdout_data, stderr_data) = p1.communicate(input=self.stdin)
(stdout_data, stderr_data) = p1.communicate()
self.stdout['output'] = "Conversion successful. Use the embed filter to display your ressource in the page"
class AAFilterCrop(AAFilter):
"""
[[ http://example.org/image.jpg||crop:40x30+10+10 ]]
"""
name = "crop"
def validate(self):
sizepat = re.compile(r"(?P<width>\d+)x(?P<height>\d+)\+(?P<top>\d+)\+(?P<left>\d+)", re.I)
match = sizepat.match(self.arguments)
if match:
self.parsed_arguments['width'] = match.groupdict()['width']
self.parsed_arguments['height'] = match.groupdict()['height']
self.parsed_arguments['top'] = match.groupdict()['top']
self.parsed_arguments['left'] = match.groupdict()['left']
return True
def run(self):
if not os.path.exists(self.stdout['local_path']):
cmd = 'convert -crop %sx%s+%s+%s %s %s' % (self.parsed_arguments['width'],
self.parsed_arguments['height'],
self.parsed_arguments['top'],
self.parsed_arguments['left'],
self.stdin['local_path'],
self.stdout['local_path'])
p1 = subprocess.Popen(cmd.split(" "), stdout=subprocess.PIPE,
stdin=subprocess.PIPE)
#(stdout_data, stderr_data) = p1.communicate(input=self.stdin)