Who thinks TivoHD MRV times will ever match S3?

[ghken]

I have read the FAQ's and various threads on MRV transfer times and know that S3 --> S3 is currently the speed champ. I've also read tidbits here and there about TivoHD speeds possibly improving down the road with a software update.

For those in the know, is this MRV speed discrepancy between the S3 and THD ever likely to change, or is the hardware platform of the TivoHD the limiting factor?

I'm currently in the middle of upgrading my Series 2 environment with HD models. I have already replaced one S2 with an S3. Just trying to decide whether my second box should be an S3 as well. If there's hope that THD MRV rates will eventually improve, I would feel more comfortable in picking one up. Otherwise, I will probably pay the premium for another S3.

[GumboChief]

If MRV mattered to me, I would get the S3.

[AbMagFab]

For those in the know, is this MRV speed discrepancy between the S3 and THD ever likely to change, or is the hardware platform of the TivoHD the limiting factor?

The THD and S3 are virtually identical as it relates to MRV. MRV entails the CPU, the HDD, memory, the network, and eSATA (and software). The THD is the same or better in all these categories.

So why is it slower? Who knows. Extremely disappointing if you ask me. It's basically unusable for me, since it's slightly slower than real-time, meaning I'd have to start MRV'ing a 1-hour show for 15-20 minutes before watching it. Easier to just record it twice - making MRV useless.

The only explanation that makes any sense is some hardware driver issue, but I can't imagine what that could be, with everything else working just fine.

If it matters to you, you should get S3's. I would have had I known of this issue before, but I was already set up with all my S3's and THD's before MRV was released... darn...

[mikesown]

TiVo can remedy the problems -- they just choose not to spend their resources on TTG/MRV development. Several people managed to install custom "jumbo frame" drivers to increase brute(direct-off-disk-to-network) transfer rate speeds to about 70-80mbit/sec compared to the 10-30mbit/sec now. Complain to TiVo; it's not the hardware that's limiting the speeds -- it's the drivers.

[AbMagFab]

TiVo can remedy the problems -- they just choose not to spend their resources on TTG/MRV development.

That's an odd statement, since it was the major reason for the last release. It sure seemed to me (and most I would imagine) that the bulk of the time they spent on the last release was for HD TTG/MRV.

[bicker]

Well, not really. Remember, that TiVo was also working on the port of TiVo software to the Motorola DCT/DCH platform. And perhaps a whole bunch of things we don't even know about. So mikesown's point presumably was they didn't expend as much on TTG/MRV as he would have liked them to.

[AbMagFab]

they didn't expend as much on TTG/MRV as he would have liked them to.

That's certainly a true statement, especially since it's more subjective.

[HDTiVo]

From what we know of the TiVo HD hardware, there isn't any reason to think it couldn't transfer as fast or faster than an S3. But we don't know everything.

I figure TiVo will increase the MRV & TTC&G speeds on the TiVo HD substantially over time. The S3 is too slow also, and I hope they increase that as well.

[CharlesH]

Well, not really. Remember, that TiVo was also working on the port of TiVo software to the Motorola DCT/DCH platform. And perhaps a whole bunch of things we don't even know about. And perhaps on the Tuning Resolver to get TiVo to work with SDV :)

[nhaigh]

The S3 is too slow also, and I hope they increase that as well.

Why do you say that? Two S3's will MRV in real time and allow you to FF through the ads. It may be possible to make it faster but its not too slow now.

[bkdtv]

Why do you say that? Two S3's will MRV in real time and allow you to FF through the ads. It may be possible to make it faster but its not too slow now.I agree that MRV on the S3 is more than fast enough.

He was probably referring to TTG, which maxes out at 15Mbps on the S3. With various "hacks," people are transferring from the S3 to their computer over 50Mbps, I believe.

[Scott D]

If this speed issue between the two TiVos is a software related issue, I would tend to believe that this will be corrected in the near future.

I say this based on DirecTiVo S2 receivers. DTV said they would activate the USB ports and never did and thus I can't trust DTV, among other things, to their word!:D

[TexasGrillChef]

TiVoPony at one point in time I know said they are fully aware of the issue of the TiVoHD having slower transfer speeds then the S3.

I also believe he said this was due to "overhead" issues in the software & that they ARE working on speeding TiVoHD Transfer speeds up in FUTURE software releases.

I do admit and acknowlege that this might not be a quick enough remedy for some people.

Just be paitient, it will come.

TGC

[CrispyCritter]

TiVo can remedy the problems -- they just choose not to spend their resources on TTG/MRV development. Several people managed to install custom "jumbo frame" drivers to increase brute(direct-off-disk-to-network) transfer rate speeds to about 70-80mbit/sec compared to the 10-30mbit/sec now. Complain to TiVo; it's not the hardware that's limiting the speeds -- it's the drivers.Do you have a pointer to that? You can get that speed with unencrypted shows on hacked units, but I was unaware that people could get that speed on encrypted (normal) shows. My impression was handling the encryption was still the base cause of the slowness.

[ghken]

Thanks for the replies everyone. Kind of reinforces the idea that it's still a roll of the dice to bank on TivoHD getting quicker. All logic says it should be at least as quick as the S3, but because it isn't and there isn't clear consensus on why, I'll look to pick up a second S3.

Although it is kind of funny that I'm so hung up on the MRV speeds. I initially went through quite a bit of effort to get my S2's setup for faster than realtime transfers using wireless G bridges, and if I look back on how much I've actually used the feature... hmmm - too depressing to think about.

Looks like I'm repeating the cycle for my HD upgrade, as I hunt down NIM100's on ebay, obsess over broadcast bit rate calculations, and ponder the mystery of TivoHD's MRV shortcomings.

[HDTiVo]

Why do you say that? Two S3's will MRV in real time and allow you to FF through the ads. It may be possible to make it faster but its not too slow now.

Yes it is.

[bkdtv]

Yes it is.What would be the benefit of making it faster? What can you do with "faster" MRV that you can't do with two wired Series3 DVRs now?

[HDTiVo]

What would be the benefit of making it faster? What can you do with "faster" MRV that you can't do with two wired Series3 DVRs now?

Not set my S3s to SD or blank channels to get an adequate transfer rate.

Skip further ahead in programs to points of interest. Sports & news for example. In particular, football can not be transfered as fast as it can be watched; other sports can also have this issue.

Transfer multiple programs which I may want to sample before deciding which to watch.

Transfer programs to/from S3s simultaneously at adequate speeds.

Free up S3s to do other transfers sooner.

Dozens of other things I will never think of but others will.

Why do you say that? Two S3's will MRV in real time and allow you to FF through the ads. It may be possible to make it faster but its not too slow now.

The above is an example of adequacy for the most basic of MRV uses in a single person household.

[nhaigh]

The above is an example of adequacy for the most basic of MRV uses in a single person household.

Not sure if that is totally fair. There are three of us and we have four TiVo's, two of which are S3's. One is my main S3 and has expanded storage and it records 90% of what we watch in the family. We use MRV a great deal mainly to watch HD content recorded on the main S3 using the S3 in the bedroom. My daughter uses MRV mainly to pull disney programs from one of the S2's (with an STB) onto her S2 which does not have any digital channels.

Not set my S3s to SD or blank channels to get an adequate transfer rate.

Not sure what you mean here but I'm not that up to speed on the techicalities. From a users perspective typically both my S3's are recording on both channels, usually in HD, either beacuse of season passes or suggestions. We often MRV between them with all four tuners recording and never have a problem with catching up to the buffer. Is this not the norm?

Skip further ahead in programs to points of interest. Sports & news for example. In particular, football can not be transfered as fast as it can be watched; other sports can also have this issue.

I conceed that Football may be a problem given there is so little sport actually occuring in the three hours its on. I'm fairly sure I'd hit the buffer if I was to watch it.

[HDTiVo]

Not sure if that is totally fair. There are three of us and we have four TiVo's, two of which are S3's. One is my main S3 and has expanded storage and it records 90% of what we watch in the family. We use MRV a great deal mainly to watch HD content recorded on the main S3 using the S3 in the bedroom. My daughter uses MRV mainly to pull disney programs from one of the S2's (with an STB) onto her S2 which does not have any digital channels.



Not sure what you mean here but I'm not that up to speed on the techicalities. From a users perspective typically both my S3's are recording on both channels, usually in HD, either beacuse of season passes or suggestions. We often MRV between them with all four tuners recording and never have a problem with catching up to the buffer. Is this not the norm?



I conceed that Football may be a problem given there is so little sport actually occuring in the three hours its on. I'm fairly sure I'd hit the buffer if I was to watch it.

Well, you gave an example of your household usage which is equivalent to two seperate single person households using MRV in the least challenging of ways. Try MRV'ing a show to the main TiVo from the bedroom while also going the other way, so you and your wife are watching two different things. Try letting your daughter MRV something from the main TiVo while it is MRV'ing to your bedroom.

Go anywhere beyond your type of usage and you realize that MRV is minimally adequate only for watching from start to finish standard shows where you only skip commercials that occur every 10 or 15 minutes.

[kb7oeb]

With two THDs MRV is too slow to watch in real time, even lower bitrate channels like FoxHD run the buffer out on 100MB ethernet. I don't know why they encrypt shows that are transmitted unencrypted over the air.

[AbMagFab]

With two THDs MRV is too slow to watch in real time, even lower bitrate channels like FoxHD run the buffer out on 100MB ethernet. I don't know why they encrypt shows that are transmitted unencrypted over the air.


It's not the encryption. S3->S3 is about 1.5x real time. This is simply a THD software/driver issue that Tivo chose not to fix in this last release (they were clearly aware of it given the comments on this board at release time).

(THD and S3 use the same encryption, so there's no transcoding going on to slow things down, like S3/THD <-> S2.)

[JamieP]

...
The THD and S3 are virtually identical as it relates to MRV. MRV entails the CPU, the HDD, memory, the network, and eSATA (and software). The THD is the same or better in all these categories.
...I disagree with this assessment, particularly with respect to the memory system. In my experience, the TiVoHD has about half the memory bandwidth available to the CPU as compared to the original Series3. For example, "hdparm -T /dev/hda" achieves 16 MB/sec on the TiVoHD and 33 MB/sec on the Series3. (In spite of what it looks like, this is really a memory bandwidth benchmark rather than a disk benchmark since it is measuring buffer cache reads rather than physical disk reads.)

My conjecture is that the memory performance difference relates to the fact that the TiVoHD is a more integrated design with the main memory serving more purposes than it does in the Series3. For example, the Series3 has independent memory for the decoder, while on the TiVoHD, main memory is used for that purpose.

In the past, we saw a similar difference between the 540 and 240 Series2 models, for similar reasons.

The bottom line is that you get what you pay for, and there is a price to be paid for cost reduced designs.

[HDTiVo]

My conjecture is that the memory performance difference relates to the fact that the TiVoHD is a more integrated design with the main memory serving more purposes than it does in the Series3. For example, the Series3 has independent memory for the decoder, while on the TiVoHD, main memory is used for that purpose.


But the THD has two seperate banks of memory each equal in size to S3's total memory. Where does that take the analysis?

[AbMagFab]

I disagree with this assessment, particularly with respect to the memory system. In my experience, the TiVoHD has about half the memory bandwidth available to the CPU as compared to the original Series3. For example, "hdparm -T /dev/hda" achieves 16 MB/sec on the TiVoHD and 33 MB/sec on the Series3. (In spite of what it looks like, this is really a memory bandwidth benchmark rather than a disk benchmark since it is measuring buffer cache reads rather than physical disk reads.)

My conjecture is that the memory performance difference relates to the fact that the TiVoHD is a more integrated design with the main memory serving more purposes than it does in the Series3. For example, the Series3 has independent memory for the decoder, while on the TiVoHD, main memory is used for that purpose.

In the past, we saw a similar difference between the 540 and 240 Series2 models, for similar reasons.

The bottom line is that you get what you pay for, and there is a price to be paid for cost reduced designs.

I don't think that test alone says much at all. The memory architecture is different, and I believe (although I'm not sure - this was analyzed very early in the THD life) there are now two 16MB channels instead of one 32MB channel (or whatever they are). I think the issue might be that the THD is currently utilizing only one of the channels.

In any case, the transfer rate required for a show is about 2-3MB/s. So with two shows recording, one playing, and one MRV'ing, you'd only need about 12MB/sec anyway (and frankly, with all that going on, feel free to slow down my MRV). But currently, a THD with *nothing* going on (i.e. tune both tuners to dead channels), you can't get MRV to go at more than about 1.5MB/sec (12-15Mb/s).

(I might have screwed up some math here- but the points are valid. Jump on me for my math and I'll correct it.)

[sfhub]

In any case, the transfer rate required for a show is about 2-3MB/s. So with two shows recording, one playing, and one MRV'ing, you'd only need about 12MB/sec anyway (and frankly, with all that going on, feel free to slow down my MRV). But currently, a THD with *nothing* going on (i.e. tune both tuners to dead channels), you can't get MRV to go at more than about 1.5MB/sec (12-15Mb/s).

(I might have screwed up some math here- but the points are valid. Jump on me for my math and I'll correct it.)
That analysis is essentially describing disk transfer rates. You haven't accounted for encryption on storage and decryption on playback both of which are memory operations if the algorithms are implemented in software. You haven't accounted of going from memory to the ethernet interface. Also one doesn't know which operations are DMA and which aren't. If the memory architecture is shared among different components without dedicated segments sometimes there is contention.

It is really hard to tell where the bottleneck is without profiling the system. Is it the bus, the memory channel, processor, CPU scheduling, bug in software, etc. etc. If you see which system calls the time is being spent on, then it becomes much clearer.

[bkdtv]

I disagree with this assessment, particularly with respect to the memory system. In my experience, the TiVoHD has about half the memory bandwidth available to the CPU as compared to the original Series3. For example, "hdparm -T /dev/hda" achieves 16 MB/sec on the TiVoHD and 33 MB/sec on the Series3. (In spite of what it looks like, this is really a memory bandwidth benchmark rather than a disk benchmark since it is measuring buffer cache reads rather than physical disk reads.)Is this reading from the file cache or disk buffer? The Series3 has a direct SATA connection whereas the TivoHD goes through a SATA port multiplier.

My conjecture is that the memory performance difference relates to the fact that the TiVoHD is a more integrated design with the main memory serving more purposes than it does in the Series3. For example, the Series3 has independent memory for the decoder, while on the TiVoHD, main memory is used for that purpose.The Series3 and TivoHD are no different in this regard.

The Series3 uses the BCM7038 CPU/Decoder with 128Mb DDR400 RAM. The TivoHD uses the BCM7401 CPU/Decoder with 256Mb DDR400. Both chips have a 64-bit DDR400 memory interface.

The Series3 also has a BCM7411 decoder with another 128Mb DDR400 bank, but that decoder and that memory bank is not yet in use.

The bottom line is that you get what you pay for, and there is a price to be paid for cost reduced designs.DirecTV recently transitioned its DVRs from the BCM7038 (HR20) to the BCM7401 (HR21) and its STB designs from the BCM7411 to the BCM7402. The BCM7402 is the BCM7401 without the SATA controller. From the few reports I've read, performance is improved on the new DirecTV boxes.

DirecTV is working through their own share of bugs with the BCM7401-based HR21.

Below is a comparison of the Linux console boot output for the TivoHD vs Tivo Series3. I highlighted the major differences and summarized them below.

TivoHD
CPU revision is: 00020000
Primary instruction cache 32kb, linesize 16 bytes (2 ways)
Primary data cache 32kb, linesize 16 bytes (2 ways)
Fusion RAC config: Off 0x00000000
Linux version 2.4.20 (build@buildmaster70) (gcc version 3.3.4) #1 Thu Jul 26
Determined physical RAM map:
memory: 0fe22000 @ 001dd000 (usable)
...
Memory: 128220k/260232k available (1237k kernel code, 132012k reserved, 65k data, 88k init, 0k highmem)
Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
Inode cache hash table entries: 16384 (order: 5, 131072 bytes)
Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
Buffer-cache hash table entries: 16384 (order: 4, 65536 bytes)
Page-cache hash table entries: 6553
...
Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Broadcom BCM7038IDE: IDE controller on PCI bus 01 dev 00
Broadcom BCM7038IDE: chipset revision 0
Broadcom BCM7038IDE: 100% native mode on irq 49
ide0: BM-DMA at 0x0300-0x0307, BIOS settings: hda:DMA, hdb:DMA
ide1: BM-DMA at 0x0308-0x030f, BIOS settings: hdc:pio, hdd:pio
hda: WDC WD1600AVBS-63SVA0, ATA DISK drive
ide0 at 0x200-0x207,0x242 on irq 49
svwks_ide_dma_speed: hda: mode 0x03, speed 0x45
blk: queue 80187130, I/O limit 4095Mb (mask 0xffffffff)
...
...
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
IP: routing cache hash table of 2048 buckets, 16Kbytes
TCP: Hash tables configured (established 16384 bind 32768)
IP-Config: No network devices available.
ip_conntrack version 2.1 (2047 buckets, 16376 max) - 152 bytes per conntrack
...
Algorithmics/MIPS FPU Emulator v1.5


Tivo Series3
CPU revision is: 0001810b
FPU revision is: 0003810b
Primary instruction cache 32kb, linesize 32 bytes (2 ways)
Primary data cache 32kb, linesize 32 bytes (2 ways)
Linux version 2.4.20 (build@buildmaster52) (gcc version 3.3.4) #1 Fri Jul 14 03:00:53 PDT 2006
Determined physical RAM map:
memory: 07e27000 @ 001d9000 (usable)
...
Memory: 42612k/129180k available (1216k kernel code, 86568k reserved, 65k data, 84k init, 0k highmem)
Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
Inode cache hash table entries: 8192 (order: 4, 65536 bytes)
Mount-cache hash table entries: 2048 (order: 2, 16384 bytes)
Buffer-cache hash table entries: 8192 (order: 3, 32768 bytes)
Page-cache hash table entries: 32768 (order: 5, 131072 bytes)
...
Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
Broadcom BCM7038IDE: IDE controller on PCI bus 01 dev 00
Broadcom BCM7038IDE: chipset revision 0
Broadcom BCM7038IDE: 100% native mode on irq 49
ide0: BM-DMA at 0x0300-0x0307, BIOS settings: hda:DMA, hdb:DMA
ide1: BM-DMA at 0x0308-0x030f, BIOS settings: hdc:DMA, hdd:DMA
hda: WDC WD2500BS-55RPB1, ATA DISK drive
ide0 at 0x200-0x207,0x242 on irq 49
svwks_ide_dma_speed: hda: mode 0x03, speed 0x45
blk: queue 80180d50, I/O limit 4095Mb (mask 0xffffffff)
...
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP
IP: routing cache hash table of 1024 buckets, 8Kbytes
TCP: Hash tables configured (established 8192 bind 16384)
IP-Config: No network devices available.
ip_conntrack version 2.1 (1024 buckets, 8192 max) - 152 bytes per conntrack

The key differences appear to be in the: FPU -- Tivo Series3 has a FPU, while TivoHD has a FPU emulator;


CPU cache linesize -- TiVo Series3 has a linesize 32 bytes for its CPU instruction and data caches, while TivoHD has a linesize of 16 bytes for each.


Memory -- TivoHD has almost 3x the amount of usable memory; and


Busmastering mode of IDE1 -- What is on IDE1? TiVo Series3 has DMA, TivoHD has PIO.
If memory bandwidth on the TivoHD is less than the Series3, I suspect the CPU cache linesize is responsible for that.

From the console logs, it looks like much of the TivoHD's memory is currently unused under Linux, even after accounting for the 33Mb system cache.

[Scott D]

Is this reading from the file cache or disk buffer? The Series3 has a direct SATA connection whereas the TivoHD goes through a SATA port multiplier.

The Series3 and TivoHD are no different in this regard.

Sooooo, are you saying the TiVo HD is a slight downgrade from the S3 model?
Also, to my understanding, is it true that the first generation of S3 TiVo's have an IDE drive and the TiVo Hd got the SATA?

[bkdtv]

Sooooo, are you saying the TiVo HD is a slight downgrade from the S3 model?
Also, to my understanding, is it true that the first generation of S3 TiVo's have an IDE drive and the TiVo Hd got the SATA?You might want to edit your previous post to remove the quote. That's a long quote!

The TiVo Series3 and TiVoHD both have SATA hard drives, as you can see in my previous post. You may be thinking of the old HDTV Tivo from DirecTV. That did have an IDE hard drive.

The TivoHD is an upgrade in some places and could be a downgrade in others. Most of the differences we've seen so far could be driver related. As seen in the Linux console logs, the TivoHD still using most of the drivers from the Series3. For example, it uses the BCM7038IDE driver but it doesn't have a BCM7038.

Dish Network and DirecTV are using the CPU/decoder from the TivoHD in their newest DVRs.

[JamieP]

I would encourage the doubters to run benchmarks themselves. While raw cpu performance is similar between the TiVoHD and the original Series3, memory bandwith is not, and you can also see this by running timed large memcpys or running the integer stream (http://www.cs.virginia.edu/stream/) benchmarks.

MRV transfers involve many data copies, so it is naive to assume that you can add up the transfer rates for MRV, playback and recording to subtract from the total available memory bandwidth budget. At the very least, the data is copied from disk to memory, and then from memory to the network interface, doubling the data rate requirements. In practice, there are many more data copies than that.

IMO, this is not a driver issue, but a more fundamental architectural issue.

[AbMagFab]

All that says is it's a driver issue, as any benchmark tests are using the same drivers as the TivoOS.

I think it's far more likely, looking at the hardware itself, that Tivo needs to update/upgrade the drivers.

And I believe hacked units get substantially better transfer rates, pointing directly to driver/software issues, not hardware limitations.

[bizzy]

Busmastering mode of IDE1 -- What is on IDE1? TiVo Series3 has DMA, TivoHD has PIO.[/list]


Most drivers negotiate transfer modes based on the capabilities that the storage devices advertise during initialization. My bet is that there wasnt a device on the IDE1 channel of the TivoHD you got the boot log from, and it defaulted the idle channel to PIO.

[ZeoTiVo]

TiVo can remedy the problems -- they just choose not to spend their resources on TTG/MRV development. Several people managed to install custom "jumbo frame" drivers to increase brute(direct-off-disk-to-network) transfer rate speeds to about 70-80mbit/sec compared to the 10-30mbit/sec now. Complain to TiVo; it's not the hardware that's limiting the speeds -- it's the drivers.

yes. TiVo does a balancing act of the many different processes going on in the TiVo. It took a major rewrite of MRV to get it enabled on the Series 3 line. Since the S3 had more mature OS than the TiVo HD they could most likely tweak it better on that platform. Given some more maturity/tweaks for the Tivo HD it will get faster as well.

I go the TiVo HD route myself and use MRV a good bit. I am confident it will get faster on the TiVo HD as they release new updates for the TiVo HD.

[JamieP]

All that says is it's a driver issue, as any benchmark tests are using the same drivers as the TivoOS.What drivers do you think are involved in a user space memory bandwidth test?

[ZXTT95]

The caches are small, and the smaller line size of the TiVoHD implies that burst mode memory transfers are half the size of the Series3. This would reduce memory bandwidth. Also, don't forget about the impact of fetching code on the available bandwidth.

What Ethernet hardware do they have? Are they the same or could the Series3 have something with more offloads?

[sfhub]

Someone answer this question.

Where is the bottleneck, cpu, memory, or i/o.

Is it a true bottleneck, algorithm issue, or access method issue.

Can the bottleneck be worked around by doing something different, reducing contention, scheduling actions in different order, changing access mode.

So far only Jamie has provided some data points to back up the memory bottleneck theory.

There was some reference to jumbo frames earlier, but no pointers. Further jumbo frames shouldn't be necessary for 100Mbps. They could help by working around some other bottleneck in the system (like responding to TCP ACKs or other latency issues). Systems are perfectly cable of getting within 10% of theoretical 100Mbps w/o the need for jumbo frames and S3 handles MRV faster without jumbo frames, so my tendency is to think (assuming jumbo frames helps TiVo HD) they are a workaround for some other problem rather than the cause of the bottleneck. Jumbo frames are also not a good general solution because many network devices and even ethernet switches do not support jumbo frames. The main ones I know of are newer gigabit switches.

[ZeoTiVo]

The hardware as it stands could move files in far better than real time if that was all the box had to do

think of it like a tuning issue and TiVo throttles various parts to make sure it all works. MRV is down the list after real time recording from two tuners and playback on the local DVR. The memory bandwidth issue could play into this as well as they figured out how to make the real time part work better and thus allow more resources for MRV. Tweaking the various drivers for better performance would play into this as well.

[JamieP]

There was some reference to jumbo frames earlier, but no pointers. Further jumbo frames shouldn't be necessary for 100Mbps. They could help by working around some other bottleneck in the system (like responding to TCP ACKs or other latency issues). Systems are perfectly cable of getting within 10% of theoretical 100Mbps w/o the need for jumbo frames and S3 handles MRV faster without jumbo frames, so my tendency is to think (assuming jumbo frames helps TiVo HD) they are a workaround for some other problem rather than the cause of the bottleneck. Jumbo frames are also not a good general solution because many network devices and even ethernet switches do not support jumbo frames. The main ones I know of are newer gigabit switches.I'm the main proponent of jumbo frames on the tivo. Most of the discussion about them can be found on DDB, and can't be linked here. I added the support to the linux usbnet driver and ported it to the tivo.

Jumbo frames reduce per-frame cpu overhead by reducing the number of frames. The goal is to reduce cpu utilization for network transfers. Because tivos have relatively underpowered processors and memory systems, cpu utilization in network transfers is high, even with 100mbps networks. Anything that can be done to reduce CPU and memory bandwidth overhead directly affects MRV performance.

Jumbo frames do help, but it is really viable here only for the enthusiast, not for the general masses. As you point out, partly this is due to the fact that they aren't standardized and aren't supported on much of the consumer grade network equipment out there. Also, you don't want to mix MTU's on the same subnet, so if you have any devices not capable of jumbo frames, you need to wall them off on a separate subnet with a gateway machine that can route between subnets and fragment large frames for the small-MTU subnet as necessary. It's not all that hard to set up, but it's not something the target audience for a tivo is likely to want to mess with.

The original question, as I understood it, was whether the MRV performance difference between the TiVoHD and the original Series3 is a hardware or software issue. I still believe it is a hardware issue, but as a TiVoHD owner, I'll be happy to be proven wrong if/when TiVo comes out with a new software release that brings the TiVoHD MRV performance in line with the original Series3. I'm less likely to be convinced by pontification based on datasheets without benchmarks or real MRV performance results to back it up.

[ZeoTiVo]

The original question, as I understood it, was whether the MRV performance difference between the TiVoHD and the original Series3 is a hardware or software issue. I still believe it is a hardware issue, but as a TiVoHD owner, I'll be happy to be proven wrong if/when TiVo comes out with a new software release that brings the TiVoHD MRV performance in line with the original Series3. I'm less likely to be convinced by pontification based on datasheets without benchmarks or real MRV performance results to back it up.

so I know that Series 2 records a different mpeg2 format from the series 3 and there is some work for the S3 to convert to that format on MRV. Anything like that happening between the Tivo HD and Series 3? I would assume they record the same format though.

[JamieP]

so I know that Series 2 records a different mpeg2 format from the series 3 and there is some work for the S3 to convert to that format on MRV. Anything like that happening between the Tivo HD and Series 3? I would assume they record the same format though.To quote TiVoPony [source (http://www.tivocommunity.com/tivo-vb/showthread.php?p=5624163#post5624163)]:Native transfers simply mean between like-generation platforms. Series2 to Series2. Series3 to Series3. Series3 to TiVo HD.

All Series3 generation machines (including the TiVoHD) store video in the same format, but it is different than the Series2 format.

[blacknoi]

I have a S3 and a TivoHD.


When I start a MRV from TivoHD to S3, watching the programming on the S3 as its coming in, it definitely comes in FASTER than real time. I can watch the first segment of a show and by the time the first commercial block comes up, I can skip past it. This then holds true for the rest of the show (being able to skip past commercial blocks).

However, if I'm watching the TivoHD, doing an MRV from the S3, the TivoHD transfers the content in real time, or sometimes slightly LESS than real time ( I get pauses in the stream, have to wait 10 seconds or so, and then hit play again).

Forget about skipping commercials at all.

So it seems that the TivoHD can SEND faster than it can receive. I don't get that.

[AbMagFab]

What drivers do you think are involved in a user space memory bandwidth test?

Unless you've eliminated the OS in your testing, you're going through some sort of driver.

[sfhub]

I'm the main proponent of jumbo frames on the tivo. Most of the discussion about them can be found on DDB, and can't be linked here. I added the support to the linux usbnet driver and ported it to the tivo.

Jumbo frames reduce per-frame cpu overhead by reducing the number of frames. The goal is to reduce cpu utilization for network transfers. Because tivos have relatively underpowered processors and memory systems, cpu utilization in network transfers is high, even with 100mbps networks. Anything that can be done to reduce CPU and memory bandwidth overhead directly affects MRV performance.
So was the jumbo frames reducing USB CPU overhead, TCP/IP CPU overhead, or both? My impression is USB is CPU intensive, at least when comparing CPU usage with hard drive enclosures.

I ask because if it is really USB CPU overhead that is being reduced then the same performance increases might not carry over to internal ethernet interfaces. Then again if it is TCP/IP overhead, then similar gain is possible. I didn't read the DDB thread so I'm assuming some gain was made using jumbo frames.

[AbMagFab]

So was the jumbo frames reducing USB CPU overhead, TCP/IP CPU overhead, or both? My impression is USB is CPU intensive, at least when comparing CPU usage with hard drive enclosures.

I ask because if it is really USB CPU overhead that is being reduced then the same performance increases might not carry over to internal ethernet interfaces. Then again if it is TCP/IP overhead, then similar gain is possible. I didn't read the DDB thread so I'm assuming some gain was made using jumbo frames.


We already know that the S2DT is like 10x faster using the built-in ethernet port than when using a USB ethernet dongle. So there's significant gain (whatever the reason) using the internal port.

Since the S3's and THD's already have this, that gain is already, well, gotten. Hopefully something like jumbo frames will help further. If they could eek out a 30-50% improvement, then things would be fine for the THD (and amazing for the S3).

[JamieP]

Unless you've eliminated the OS in your testing, you're going through some sort of driver.Probably a pointless argument. I'm still of the opinion that the drivers are not relevant to a user space memory test that doesn't call into the kernel. I'll grant you that the drivers could have an impact if they were causing an unusually high interrupt level or something, but I've seen no evidence of that.

We already know that the S2DT is like 10x faster using the built-in ethernet port than when using a USB ethernet dongle. So there's significant gain (whatever the reason) using the internal port.In my experience, on the S2DT, S3 and TiVoHD: usb dongle without jumbo < built in ethernet << usb dongle with jumbo. The gain from jumbo frames is larger than the difference between usb and built-in nic. TiVoHD results are over on ddb in post 289774 from jkozee. Sorry, I can't link here, but a site search in google for that post number is one quick way to find it.

Since the S3's and THD's already have this, that gain is already, well, gotten. Hopefully something like jumbo frames will help further. If they could eek out a 30-50% improvement, then things would be fine for the THD (and amazing for the S3).Just turning off netfilter in the kernel build gives a 20-30% performance boost to raw network performance. It also has the potential to open up the tivo to network security exploits, so it is unlikely TiVo would be willing to do that.

[AbMagFab]

In my experience, on the S2DT, S3 and TiVoHD: usb dongle without jumbo < built in ethernet << usb dongle with jumbo. The gain from jumbo frames is larger than the difference between usb and built-in nic. TiVoHD results are over on ddb in post 289774 from jkozee. Sorry, I can't link here, but a site search in google for that post number is one quick way to find it.

Just turning off netfilter in the kernel build gives a 20-30% performance boost to raw network performance. It also has the potential to open up the tivo to network security exploits, so it is unlikely TiVo would be willing to do that.

Sounds like there's hope that just with some networking tweaks, they can improve transfer speeds.

Can you do jumbo frames with the built-in ethernet port?

[JamieP]

Can you do jumbo frames with the built-in ethernet port? bcc observed that the bcm7038 has an 11 bit field for the MTU, implying that it might support an MTU of up to 2047 bytes. Jumbo frames are typically 9000, but a step up to 2047 might help a little.

For the most part, jumbo frames are a feature only on gigabit hardware. Switches won't tend to support jumbo frames when in 100mbps mode, although some NICs might. Without a switch, you'd have to do a point-to-point link directly to a PC or another tivo.

[sfhub]

In my experience, on the S2DT, S3 and TiVoHD: usb dongle without jumbo < built in ethernet << usb dongle with jumbo. The gain from jumbo frames is larger than the difference between usb and built-in nic. TiVoHD results are over on ddb in post 289774 from jkozee. Sorry, I can't link here, but a site search in google for that post number is one quick way to find it.
Was there a data point for "built-in ethernet (with jumbo)"?

As discussed before this is probably academic because jumbo frames wouldn't be practical for many users and couldn't be a general solution.

[JamieP]

Was there a data point for "built-in ethernet (with jumbo)"?No one has done that test, and I'm not even sure it will work. It would involve modifying the bcmenet driver source (which current hardwires ETH_MAX_MTU_SIZE to 0x600 -- 1536) and compiling a custom version of the driver.