I was looking into streaming a full DVB-S transponder into individual Multicast groups, one per program into the local area network. For doing this you will need a budget card because the full featured ones are told not to pass on the full stream in case of the pseudo filter pid 8192. Trying to demux and stream a full transponder with VLC lead to ~70% CPU on a 1.2Ghz Celeron. As demuxing a MPEG-2 TS (Transport Stream) is not a really hard task i was wondering what the problem was. VLC used up to 300Mbyte memory which was also very astonishing.
So i wrote a little application providing the following features:
It does now what it should with astonishing few CPU cycles. Its not rock solid software but for the purposes i am using it for it works flawless.
This is an example for getstream DVB-S tuning informations for Astra Transponder 93 offering to stream Einslive on the local url http://localhost:8001/radio/einslive. Additionally streaming out the same programm via multicast UDP on address 239.0.71.1 and sending out SAP announcements.
There is a now a mailinglist for users and developers just to make it easier to push new releases and get patches
http://gt.owl.de/mailman/listinfo/getstreamThe easiest way to get getstream is to check out the git tree:
The other way is to download one of the tarballs here.
As Telefonica O2 closed the location in Verl, Germany i had to move the git tree. The new location is git://pax.zz.de/getstream.git
DVB-S2 via S2API is done. In time for the Fifa Soccer World Championship in South Africa the S2API is ready. I ripped of the multiproto support completely. So when using multiproto you need to take an older release. The S2API is currently only used when tuning on a S2 Adapter and there might be corner cases with a S2API enabled getstream on a non S2API Capable Kernel. Please send but reports.
Getstream is still alive. Not getting loads of new features but its getting a bugfix here and there. The mailinglist is active. The next big thing will be S2API which getstream currently does not handle.
It happened a lot - gestream got its git tree. Frederik Kriewitz sent in a SAP patch. I created a mailinglist (see above) which is open now ... I have more patches around which i'd like to incorporate and need to get into shape.
Patches for getstream start flowing in. One immediatly applied is the diseqc rewrite by by Romolo Manfredini. I now cleaned up my local git tree and put it up for everyone for easier development. Another fix in here for VLC 0.9.2 which seems to be picky concerning hex session ids in the SDP/SAP.
Just a small bugfix release. One is that vlc 0.9.2 got picky about the ordering of SAP/SDP lines. I dont think the RFC says in which order thay need to be in the SAP packet but anyway. Moved the o= line to the top so that VLC will be happy again. The other is concerning adaption field for section reassembly. It seems packets with adaption field should never have worked at all as the adaption field size field would not be accounted for. The next is a possible hang if a section would match the TS_PACKET_SIZE maximum payload e.g. consume the full TS packet. This could have led to a getstream hang.
In case you were wondering why ProSieben/Sat.1 has strange hangs see this posting: Solved: "late picture skipped" with some DVB Channels
More multiproto issues. Manu changed the API in an incompatible way be introducing an mandatory IOCTL to be called and removed some fields from the dvbfe_info struct. This getstream should compile against multiproto beyond 2008-03-09.
More Multiproto fixes. The Multiproto API is inconsistent and broken in some respects (DVBFE_GET_INFO sets the Delivery System, DVBFE_SET_PARAM ignores the delsys parameter). I hope multiproto never makes it upstream in its current shape, but its currently the only DVB-S2 capable API. This release also contains an HTTP streaming fix which could lead to a remote triggerable crash.
Got my DVB-S2 Card and soon made progress in S2 support with the multiproto api. Thank goes to Karel Tuma for the first version. I havent been able to verify the results myself as the multiproto tree is currently broken for my card. Others have reported success so i am awaiting feedback on the DVB-S2 code. An interesting point to note is that even the first tuning ioctl fails for me as long as i dont run "scan" beforehand so some state is beeing cached in kernel.
Long time no release. Meanwhile i got a lot of patches for DVB-S2 but i still cant test. Yesterday i ordered myself a Skystar HD so hopefully the next release in a couple of days will contain S2 support. The release now contains bug fixes for an early startup crash in the PSI reassemby. A possibility to configure the output udp and rtp local address to bind to a specific address. I moved most of the socket handling to helper functions. There is no point in dealing with sockaddr_in structs everywhere. The SAP announcements for RTP got fixed and thank goes to Sergei Gurjev who reported (and sent a patch for) a Multicast problem where getstream did not tell the kernel correctly to send out IGMP joins for multicast groups. So all of you who had problems with random multicast problems please retry ... Also Sergei send me a patch for DeCSA which i havent applied yet. I was thinking of a more generic filter setup where one can decide to drop static pids, scramble or descramble frames and probably one day have a SoftCam in place. I also started to dig a little deeper into RTSP but nothing to show yet ...
I got multiple reports of problems with the stuck detection. Cards getting resetted without getting stuck. I am a bit helpless here as the code is for dummies. I guess this will be mine "Brown Paperbag". Another report with problems with symbol rates != 27500000 which is a bit odd. I added an ioctl FE_GET_INFO and dumped that info into LOG_DEBUG which might help getting this bug understood. Please check this release and send in bug reports with the "-dd" output. Maybe i need to add more debugging into the stuck detection.
Fixed some namespacing in our internal structs. Dropped the special case for the PAT where we let the kernel internal section reassembly do the work. We have a section reassembler so let it do its work. Did more work on the Flexcop IRQ Stop workaround. From further conversation i found that simply retuning does not help but one rather needs to drop all pids aka filter and reaquire them.
Rewrote parts of the frontend tuner - drop the thread and use some timer and event handling on the frontend filedescriptor. Add a flexcop aka Skystar2 workaround. There seems to be a hardware bug which causes the card to lock up after some hours of operation. In Kernel 2.6.13-rc3 a patch was added resetting some registers on retuning which resets the card. getstream now monitors the dvr output and retunes if we didnt have any packets for 5 seconds.
After getting a hint that multiple adapter config does not work i had a deeper look at it did the last few steps to actually make it work. As i pass references to an adapter struct everywhere there only were some config parsing issues and some static variables for libevent which cause no events to occur on the first adapter.
More work in progress. I fixed SAP in getstream2 so now it'll be able to stream UDP and send SAP/SDP which will be understood by VLC.
The last months did not pass unnoticed. I rewrote large parts of getstream concerning the demux, PSI handling, PAT and PMT parsing. This is more or less a "Work in Progress" but i put up a tarball for the interested. The config format changed slightly and so did the internal workings. Basically getstream2 will be able to stream multiple programs and multiple static pids into the same groups. I did the rewrite because the EIT (Event Information Table) witht he EPG informations is sent on a static pid and i'd like to stream this into a PIPE. Missing in getstream2 is currently the CSA encryption. I plan to implement CSA and other pid drop features in a generic "filter" part. Especially i'd like to be able to drop DSM-CC Pids containing MHP informations. Another outcome was that getstream2 uses approx. half the CPU time getstream uses so its worth trying anyway.
Yet another release. This fixes a problem with PID expire. Also the awaited PIPE stream output is now available. It can be used to postprocess MPEG streams locally or to easily dump it into a file via cat. Actually getstream ran for approx 5-6 Weeks continously at work without a single hickup.
After 2 Month another small fix concerning the PCR pids. We were generating the PCR pid with some more reserved bits and thus calculating it wrong. Also some shifts were wrong so PCR pids below 256 only should have worked.
Had some time between 4 and 7 in the morning and had a look at the "thunderstorm" problem. The problem shows as once the LOCK has been lost on the transponder the card never reaquires it without tuning in. So i made a thread which permanently monitors the fe events. If the fe event doesnt show FE_HAS_LOCK for 5 seconds we will try to retune the frontend.
Long time no release. Took the time to fix some demux stuff. getstream now will automatically try to switch budget-mode off in case it fails to set up. Fixed also some things concerning full featured mode with respect to joining pids. This is mostly untested and i am interested to get reports.
FIFA World Soccer Championship is over and interest for streaming decreases a bit ;). Got a patch from Sean Lowry who found the cause for the audio problems. Getstream was not sending the "Program Clock Reference" pid so it seems some applications or audio encodings actually need it. Added non budget card mode which explicitly joins the pid. This should help FlexCop II aka b2c2 Card owners which reported huge CC problems. There is now a SNR/BER monitor which dumps the values to stdout every 5 seconds. This is suboptimal as i'd like only to complain when something breaks. I added a simple program "simplecheck.c" which can parse a raw TS stream and show TEI and/or CC problems. All over some bugfixes and cleanups.
This should be major step for stability. Fixed more segfaults caused by broken PMT packets coming downstream. Now getstream checks for the TS Packet "Transport Error Indicator" bit when the DVB card discovers one or more uncorrectable bit errors. Also validate pointer and offsets into the PMT a bit more. Ignore it instead of crashing. Sean Lowry sent a patch for VLCs Playgroup support. Now you can form Bouquets in the VLC playlist using SAP. Sean also send a patch for encrypting the output stream with the DVB-CSA, ready to build CA on your IPTV Platform. I also rewrote the demux handling that getstream handles PIDs in multiple output streams. Hopefully this fixes the "No Audio" problem i was reported.
After flying a couple of rounds concerning broken PMTs and resulting crashes i finally fixed the random segfault. The whole PMT parsing got much more robust only touching the PMT when length seems sane and the CRC matches. There is a change in the HTTP streaming sending "Pragma: no-cache" and "Cache-Control: no-cache" now. This came with restructuring the libhttp stuff.
Another update-Fixed a segfault in the pidtable (need to retain the backtraces next time). Did some dvb-t configuration issue stuff to actually allow "auto" and "none" on the settings. Fixed a bug in overrunning the PMT while parsing. Started to write something about the configuration possbility in the README. Needs to get a lot more.
Introduced some nasty bugs concerning PAT CRCs and beeing ignorant about multi TS Packet PMTs and then requiring correct CRC. This has all been fixed and a lot more concerning demux.c. Hopefully this will lead to a working getstream again. Please test.
Got mail from Stephen Gardner mentioning H.264 HDTV success in London. Added the PID stream types to demux. While at it Stephen mentioned bad PIDs added to the demuxer on a weak signal. As i was debugging i noticed the CRC32 code beeing broken so i added new code from the Linux Kernel. Now demux checks for bad PMT CRC and ignores broken packets. Some more sanity checks added hopefully i didnt break anything. As noone noticed i guess the PAT CRCs i sent out were broken too. Need to check.
Another release. Optimized the config file parser and told it to know the difference between IPv4 and IPv6. Tero Pelander submitted a patch for DVB-C support. Fixed a SAP crash caused by the stream output restructuring. Increased PID timeouts in demux.
Put my latest CVS state up now. Fixed some bugs in the libhttp code which could cause hangs or crashes in case of accidental hangs on the client or flaky network. Also i added a guardian thread which basically monitors if the libevent loop in the main program is still triggering events. If not we'll send a SIGABRT and when you are lucky and have set a core ulimit we'll have something to debug afterwards.
Finally we have DVB-T in our Region. This was done in a quick hurry for the coming Football World Championship. DVB-T startet this morning and i just added support for it to getstream. Besides i fixed some bugs and added HTTP/1.0 i.e. non "Transfer-Encoding: chunked" streaming.
Loads of work done. Basically added a config file for the stream output options. It was to nasty to put everything on the command line and parse it afterwards. Added RTP and HTTP streaming. Beginnings of RTCP are there although vlc is not playing as i thought when sending SAP announcements for a Unicast address offering RTCP/RTP. This release is definitly a major rework and lots of work remains to be done. Next thing to do is to get rid of the .dvbrc as noone seems to be able to generate one. On the 29th of May i'll finally be able to receive DVB-T around here so i'll try my best to put DVB-T support in. Then i'll try to do RTSP streaming which i also needed the basic http code for. Thanks for all the patches i got.
Reports of hardcoded path in reading dvbrc. *Urgs*. Fixed. Also a bug appeared that getstream would hang if no .dvbrc is existant. Fixed also. Forgot also to license the code. Put the GPL in it.
Initial release. Functionality works. 5 top most important transponders are streamed for the last 3 days without hickups.
Start of project. Wrote lex/yacc parser for parsing the .dvbrc. Initial opening of the demux0 and frontend0, tuning and basic libevent TS stream reading.