nlnet_2019_video.mdwn

   1 # NL.net proposal
   2
   3 ## Project name
   4
   5 The Libre-RISCV SoC, Video Acceleration
   6
   7 ## Website / wiki
   8
   9 <https://libre-riscv.org/nlnet_2019_video>
  10
  11 Please be short and to the point in your answers; focus primarily on
  12 the what and how, not so much on the why. Add longer descriptions as
  13 attachments (see below). If English isn't your first language, don't
  14 worry - our reviewers don't care about spelling errors, only about
  15 great ideas. We apologise for the inconvenience of having to submit in
  16 English. On the up side, you can be as technical as you need to be (but
  17 you don't have to). Do stay concrete. Use plain text in your reply only,
  18 if you need any HTML to make your point please include this as attachment.
  19
  20 ## Abstract: Can you explain the whole project and its expected outcome(s).
  21
  22 The Libre RISC-V SoC is being developed to provide a privacy-respecting
  23 modern processor, developed transparently and as libre to the bedrock
  24 as possible.
  25
  26 One of the main "hardware accelerated blocks" of any processor intended for user applications is Video Encode and Decode. This usually means an opaque, proprietary piece of hardware, and it usually comes with proprietary firmware as well.
  27
  28 In a privacy-respecting world neither of these are acceptable, therefore the goal is to develop - in an iterative fashion - not just the software but the actual hardware instructions (similar to ARM NEON) which, if fully integrated into libswscale, ffmpeg, gstreamer and other software, would make RISC-V a truly commercially competitive peer of ARM and x86 systems when it comes to video acceleration.
  29
  30 With such capability freely available, there would thus be no opportunity and no excuse for the inclusion of spying hardware blocks or coprocessors in modern RISC-V processors.
  31
  32 # Have you been involved with projects or organisations relevant to this project before? And if so, can you tell us a bit about your contributions?
  33
  34 Luke Leighton is an ethical technology specialist who has a consistent
  35 24-year track record of developing code in a real-time transparent
  36 (fully libre) fashion, and in managing Software Libre teams.  He is the
  37 lead developer on the Libre RISC-V SoC.
  38
  39
  40 # Requested Amount
  41
  42 EUR 50,000.
  43
  44 # Explain what the requested budget will be used for?
  45
  46 The tasks, which will need to be iteratively applied, are as follows:
  47
  48 * to identify closely the key areas in video decode, across a wide range of algorithms, where a non-accelerated processor spends considerable CPU time and power consumption.
  49 * to propose and then evaluate the instructions which, if included in RISC-V, would speed up video decode and reduce power consumption to within commercially competitive levels.
  50 * to simulate those proposed instructions and confirm their viability
  51 * to implement those instructions in actual hardware, for running in an FPGA
  52 * to follow through with the upstream submission and acceptance of customisation of relevant software libre video decode projects and toolchains.
  53
  54 This needs to be done iteratively because it is only when a certain high level of functionality is reached (FPGA, full simulation) will it be possible to properly determine if the proposed instructions actually meet the requirements. It may turn out that further optimisation is needed.
  55
  56 # Does the project have other funding sources, both past and present?
  57
  58 The overall project has sponsorship from Purism as well as a prior grant
  59 from NLNet.  However that is for specifically covering the development
  60 of the RTL (the hardware source code).
  61
  62 There is no source of funds for the work on Video ISA development (only for its hardware implementation) or the follow through work which involves getting support for that ISA extension upstream in relevant software (ffmpeg, vlc, gstreamer) and toolchains (gcc, llvm, binutils)
  63
  64 # Compare your own project with existing or historical efforts.
  65
  66 There do exist on opencores a number of video encode and decode blocks: these are typically MPEG and h.264. However, the problem is that these are dedicated blocks, specific to those algorithms. They do not help with h.265, Theora, Dirac, vp8, vp9 and anything else that may come out.
  67
  68 Any instructions which exist for OpenRISC1200 or MIPS will not help either, because the instructions need to be evaluated specifically for RISC-V.
  69
  70 So the answer is: this initiative is unique, and there are no peer projects: the RISC-V initiative itself is too recent.
  71
  72 ## What are significant technical challenges you expect to solve during the project, if any?
  73
  74 The actual process is technically quite straightforward, and given that ffmpeg and so on are quite well established and platform independent,
  75
  76 ## Describe the ecosystem of the project, and how you will engage with relevant actors and promote the outcomes?
  77
  78 LIP6 have their own mailing list for the (transparent) discussion of
  79 issues related to coriolis2: <alliance-users@asim.lip6.fr>.  The Libre RISC-V
  80 SoC has a full set of resources for Libre Project Management and development:
  81 mailing list, bugtracker, git repository and wiki - all listed here:
  82 <https://libre-riscv.org/>
  83
  84 In addition, we have a Crowdsupply page
  85 <https://www.crowdsupply.com/libre-risc-v/m-class> which provides a public
  86 gateway, and heise.de, reddit, phoronix, slashdot and other locations have
  87 all picked up the story.  The list is updated and maintained here:
  88 <https://libre-riscv.org/3d_gpu/>
  89
  90 # Extra info to be submitted
  91
  92 * <http://libre-riscv.org/3d_gpu/>
  93 * <https://www-soc.lip6.fr/equipe-cian/logiciels/coriolis/>
  94 * <https://nlnet.nl/project/Libre-RISCV/>
  95 * <https://chips4makers.io/blog/>
  96