TiVo Community
TiVo Community
TiVo Community
Go Back   TiVo Community > Main TiVo Forums > TiVo Home Media Features & TiVoToGo
TiVo Community
Reply
Forum Jump
 
Thread Tools
Old 06-02-2012, 05:47 PM   #1
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Big Grin built debian server to host tivo hd recordings, next steps?

Woo hoo!! Installed debian on my new home-built off a usb drive (that was a challenge in itself); everything's cool! In fact I'm writing to you now from Iceweasal on the new server.

I know I'd like the server to run headless and I know that involves running xdmcp to allow me to bring up the desktop remotely, say on my windows laptop; and I know I need to share the filesystem(s) on the server, but I could use a hint how to accomplish all that.

Meantime I'll keep digging.

Thanks!
markmarz is offline   Reply With Quote
Old 06-03-2012, 01:07 AM   #2
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Since you are posting from IceWeasel, I take it you are running a desktop already. That being the case, you already have X up and running. One thing (I don't recall) are you running an Intel or AMD system? I don't know if it has been fixed, but there was a small bug that caused some Intel based systems to lock up on boot if there was no display. It's a fairly easy fix if it happens, so report back if you cannot bring up your system without a monitor attached while running X.

The following assumes you have a secure network, specifically a NAT firewall providing internet access. If not, we will need to talk about implementing a firewall on the server. I recommend the external firewall, but if you don;t want to or for some reason are unable to implement the external firewall, ping back and we can secure your system before moving forward.

Meanwhile, the first step is to make sure the server has a fixed IP. A dynamic IP and servers do not mix well. Your clients really need to know where they can find the server without a lot of contortions.

Next you need to open up the system to respond to XDMCP requests.

Finally, you should have ssh enabled.

OK, to get started, open up a terminal session. Switch to the root user identity (please, no grumbles from paternalistic Linux types) by typing `su` and entering the root password at the password prompt. Type each of the following, allowing the system to install each software:

Code:
apt-get install vim
apt-get install openssh-server
To edit a file, issue the command

Code:
vim <filename>
Vi and vim are a bit arcane, but for your purposes all you need to know is pressing the <insert> key once enters insert mode. Pressing it successive times toggles between insert and overwrite mode. To exit either editing mode, press <Esc>. To save and quit the file, (after exiting the edit mode by pressing <Esc> ) press :wq! To exit without saving, type :q!

Edit the interfaces file by typing

Code:
vim /etc/network/interfaces
and make it look something like the following (choose the IP address as you like, making sure it is not an address your router will try to set automatically):

Code:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback
address 127.0.0.1
netmask 255.0.0.0

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
address 192.168.xxx.yyy
netmask 255.255.255.0
gateway 192.168.xxx.1
auto eth0
(This assumes your gateway router is 192,168.xxx.1)
Save and quit the file by pressing <Esc>:wq!

To enable the IP so you can use it without rebooting the machine, type

/etc/init.d/networking restart (Deprecated)

Code:
ifdown eth0
ifup eth0
The next step depends on whether you are using gdm or kdm as your display manager.

If you are using gdm, then editing the /etc/gdm/gdm.conf file to contain the lines

Code:
[xdmcp]
Enable=true
and then issuing the command

Code:
/etc/init.d/gdm restart
should allow you to log in to the server with an X-server such as X-ming or Exceed Hummingbird from a networked computer. Of course you will need to install the X-server of your choice on the remote machine, first. There is a free version of X-ming available for Windows.

If you are using kdm (which I prefer) as your display manager, then edit the /etc/kde4/kdm/kdmrc file to contain the same two lines. Note there are a great many more options in /etc/kde4/kdm/kdmrc, and you might like to study them to see what other options for logging in you may want to implement. Once again, issue the

Code:
/etc/init.d/kdm restart
command to enable your changes.

Last edited by lrhorer : 06-09-2012 at 03:38 PM.
lrhorer is offline   Reply With Quote
Old 06-03-2012, 03:28 AM   #3
wmcbrine
Resistance Useless
 
wmcbrine's Avatar
 
Join Date: Aug 2003
Posts: 9,156
Quote:
Originally Posted by markmarz View Post
I know I'd like the server to run headless and I know that involves running xdmcp to allow me to bring up the desktop remotely
That's not really necessary. Unlike in Windows, in Linux you can do most of what you need to from the command line, so telnet or ssh is sufficient. On the rare occasions when I want to run an X app remotely, I just do an "ssh -X" and start the app from the command line.

Leaving an X environment running full-time on a headless server can be a waste of server resources.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
wmcbrine is offline   Reply With Quote
Old 06-03-2012, 06:18 AM   #4
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Sounds good. So ssh -X will enable me to later kick off kmttg, right?

I avoid running X full-time by not kicking off xdmcp?

Sorry for the dumb questions .. long-time programmer, been using Solaris for years and now AIX at work, but novice linux administrator. I found this site this morning: http://www.linux-tutorial.info. Seems like a good start; there's some I know, but only picked up haphazardly. I learn best by starting at the beginning.
markmarz is offline   Reply With Quote
Old 06-03-2012, 06:27 AM   #5
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by lrhorer View Post
Since you are posting from IceWeasel, I take it you are running a desktop already. That being the case, you already have X up and running. One thing (I don't recall) are you running an Intel or AMD system?
amd-64

Quote:
if you don't want to or for some reason are unable to implement the external firewall, ping back and we can secure your system before moving forward.
ping!

I've been looking around at linux security since reading your reply and realize this is going to require a whole heck of a lot of background study. I'm not adverse to having the incantations handed over w/o much of an understanding in the interests of expediency! I'm going to hold off on further steps as you suggest until security is set up. I'll be trying to learn the basics in the meantime.

Quote:
The next step depends on whether you are using gdm or kdm as your display manager.
Don't know for sure but I think GDM. I'm using LXDE, is that a clue?
markmarz is offline   Reply With Quote
Old 06-03-2012, 06:49 AM   #6
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by lrhorer View Post
apt-get install vim
apt-get install openssh-server
Well, I did just go ahead and attempted to add this software; didn't seem like that would hurt. Although vim installed fine, having a problem with openssh-server:

Code:
Media change: please insert the disc labeled
 'Debian GNU/Linux 6.0.5 _Squeeze_ - Official amd64 xfce+lxde-CD Binary-1 20120512-14:33'
in the drive '/media/cdrom/' and press enter
The sad thing is I didn't actually install from a physical cd; I installed from a cd image put to a usb stick via win32 disk imager because I don't have a cd player right now. Tried re-inserting usb stick but didn't help. I also know diddly-squat about filesystems on linux so no idea how to (say) point to the usb stick if pointing is what's needed. I will be getting a cd player in a few days; I could write the install image to it and try from there.
markmarz is offline   Reply With Quote
Old 06-03-2012, 07:16 AM   #7
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Okay, problem solved. Installed synaptic and disabled cd rom image from package sources, then installed openssh-server.
markmarz is offline   Reply With Quote
Old 06-03-2012, 05:11 PM   #8
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by markmarz View Post
Okay, problem solved. Installed synaptic and disabled cd rom image from package sources, then installed openssh-server.
Yep. The sources list is kept at /etc/apt/sources.list. I'm a little surprised the CD-ROM source was kept there, since you did a netinst installation.

I don't much like Synaptic, but it does get the job done. I much prefer kpackage, but for some bizarre reason they have decided to pull it from development. I'm using a back port. Don't worry about the file systems. Linux can take care of all that for you until you are ready to format your array (or whatever you decide to deploy). For a video server, I like XFS. It handles underlying arrays expeditiously, and it is very fast at creating and writing files. It tends to be a bit slow at deleting thousands of files, but on a video server, that almost never happens.
lrhorer is offline   Reply With Quote
Old 06-03-2012, 05:43 PM   #9
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Thanks, I will go with XFS. Seems particularly well suited to my enormous TD HD files, which should be the overwhelming majority of user files.

Quote:
ping back and we can secure your system before moving forward
In the meantime, I would like to implement a software firewall (don't want to buy an external firewall) .. is this offer still on?

Other than wireless security (but this server is wired to switch as is the pc I plan to remote into it with), and whatever firewall features are implemented by default in the DLink 655 router, I'm not aware of any other security on the network.

Quote:
specifically a NAT firewall providing internet access
Don't know, but don't think I have that. I'm reluctant to move forward until the system is secure, or at least secure enough. Or does going forward with your instructions starting with
Quote:
the first step is to make sure the server has a fixed IP
all I need to do?
markmarz is offline   Reply With Quote
Old 06-03-2012, 06:12 PM   #10
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by wmcbrine View Post
That's not really necessary. Unlike in Windows, in Linux you can do most of what you need to from the command line, so telnet or ssh is sufficient. On the rare occasions when I want to run an X app remotely, I just do an "ssh -X" and start the app from the command line.
It's true it is not strictly necessary, and I frequently spawn independent X apps in their own windows. There are some advantages to using a native Desktop manager, however. Among them are the ability to automatically launch a suite of apps on the server upon login under a single desktop, having multiple desktops, and I find operations like cut-and-paste to be much, much easier under KDE than Windows.

Quote:
Originally Posted by wmcbrine View Post
Leaving an X environment running full-time on a headless server can be a waste of server resources.
Yeah, but I don't think he is going to be doing it full time. Of course, neither way is terribly difficult. What's more, on modern systems, the use of resources by the X environment is not really aggressive compared to their capabilities. I have an XDMCP KDE session up right now on one of my servers with kmttg, five terminal sessions, kpackage, Gnome System Monitor, and KNUT client all running. One CPU is below 4%, and the other is running 26%, but only because the monthly array check is running. Ordinarily, they would both be well under 4%. Memory is 2.7G out of 4G, with 1G of swap. Normally, swap would be under 800K, but wine is using a pretty big chunk.
lrhorer is offline   Reply With Quote
Old 06-04-2012, 12:36 AM   #11
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by markmarz View Post
Thanks, I will go with XFS. Seems particularly well suited to my enormous TD HD files, which should be the overwhelming majority of user files.
Yeah, it is.

Quote:
Originally Posted by markmarz View Post
In the meantime, I would like to implement a software firewall (don't want to buy an external firewall) .. is this offer still on?

Other than wireless security (but this server is wired to switch as is the pc I plan to remote into it with), and whatever firewall features are implemented by default in the DLink 655 router, I'm not aware of any other security on the network.
Unless you have its firewall disabled, the D-Link 655 IS an external firewall. As long as the firewall on the 655 is enabled and you have your wireless segments properly secured, that is all that is needed. We certainly could implement iptables, but it really is not necessary.

Quote:
Originally Posted by markmarz View Post
Don't know, but don't think I have that. I'm reluctant to move forward until the system is secure, or at least secure enough. Or does going forward with your instructions starting with all I need to do?
Yeah, just make sure your passwords are strong, and that the local wireless network is locked down tight. You can run an insecure guest network, if you like. Disallowing remote root logins (which should be the default) is a good idea.

Last edited by lrhorer : 06-09-2012 at 03:41 PM.
lrhorer is offline   Reply With Quote
Old 06-04-2012, 12:42 AM   #12
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by markmarz View Post
Sounds good. So ssh -X will enable me to later kick off kmttg, right?
On the Linux machine, yes. Of course you can, if you choose, run kmttg on a remote (presumably windows) machine and simply have it write to the server. For your purposes, it is probably 6 of one and half a dozen of the other.

Quote:
Originally Posted by markmarz View Post
I avoid running X full-time by not kicking off xdmcp?
Yes. Perhaps more to the point, the desktop manager is then Windows, not the Linux GUI.

Quote:
Originally Posted by markmarz View Post
Sorry for the dumb questions
They aren't dumb.
lrhorer is offline   Reply With Quote
Old 06-04-2012, 12:58 AM   #13
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by markmarz View Post
amd-64
Oh, good. 'Definitely won't be a problem, then.

Quote:
Originally Posted by markmarz View Post
I've been looking around at linux security since reading your reply and realize this is going to require a whole heck of a lot of background study.
Yeah. Iptables is extremely powerful, but damnably arcane.

Quote:
Originally Posted by markmarz View Post
I'm not adverse to having the incantations handed over w/o much of an understanding in the interests of expediency! I'm going to hold off on further steps as you suggest until security is set up. I'll be trying to learn the basics in the meantime.
Well, OK, there may be some things in there you want to implement, but for now, as long as you have the firewall on the 655 set up properly and have that wireless network locked up tight, it should be fine.

Quote:
Originally Posted by markmarz View Post
Don't know for sure but I think GDM. I'm using LXDE, is that a clue?
No, I'm afraid not. I'm not very familiar with LXDE, but unless it has some requirements of which I am unaware, it should be able to run under either gdm or kdm. Indeed, until 2009, it had to run under one or the other of those. In 2009, LXDM, a display manager written specifically for LXDE was released, but I am not sure it is supported under Debian. (LXDE definitely is.)
There is a simple way to tell, though. Look at the contents of /etc/default-display-manager. It will tell you what display manager is in use.
lrhorer is offline   Reply With Quote
Old 06-04-2012, 01:51 AM   #14
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by markmarz View Post
and I know I need to share the filesystem(s) on the server, but I could use a hint how to accomplish all that.
The short answer to that is: SAMBA.

Depending on the options you selected during install, SAMBA may already be installed. You can check using Synaptic, or just type

Code:
apt-get install samba
If it is not installed automatically, I also recommend

Code:
apt-get install swat
SWAT (SAMBA Web Administration Tool) allows one to configure and administer SAMBA via web browser from any machine on the LAN.

As always, if SAMBA or SWAT are already installed and configured, apt-get will notify you of the fact.

Now the first question is, "How secure does the share need to be?" If you want anyone on the LAN to be able to read, write, and delete files on the share, things get pretty simple. If you want to restrict access, they can get rather more complicated. Note there is absolutely nothing wrong with creating more than one directory on the array, with some of the directories being shared wide open and others being locked down. For example, I created several separate wide open shares for video, music, DVD rips, etc. I also created a personal directory for myself, one for my roommate, and one for each of her daughters. Both on the server itself and on the shares, no one but me can read or write my directory, only my roommate can read and write her directory, etc.

After you install SAMBA, you will need to create SAMBA users. These are very much like regular Linux users, and indeed it may be a good idea to create the same SAMBA users as Linux users. To that end, the use of mksmbpasswd before the first time SAMBA is run will copy all the Linux users over to the /etc/smbpasswd file. The passwords will not be copied. Once you have all your Linux users created, type:

Code:
cat /etc/passwd | /usr/sbin/mksmbpasswd > /etc/samba/smbpasswd
Then use the smbpasswd utility to create the passwords for SAMBA. Here is a good basic tutorial on setting up SAMBA. Below is a copy of my /etc/samba/smb.conf file:

Code:
RAID-Server:/usr/bin# cat /etc/samba/smb.conf
# Samba config file created using SWAT
# from UNKNOWN (192.168.1.5)
# Date: 2011/08/21 08:40:03

[global]
	workgroup = HOME
	map to guest = Bad User
	guest account = lrhorer
	printcap name = cups
	disable spoolss = Yes
	mangle prefix = 6
	domain master = No
	ldap ssl = no

[Server-Main]
	path = /RAID/Server-Main/
	valid users = lrhorer, lgates, "Leslie A Rhorer"
	admin users = lrhorer
	read only = No
	guest ok = Yes

[TiVo-Music]
	path = /RAID/Music/
	admin users = lrhorer
	read only = No
	guest ok = Yes

[Video]
	path = /RAID/Recordings/
	admin users = lrhorer
	read only = No
	guest ok = Yes

[Leslie]
	path = /RAID/Personal_Folders/Leslie/
	valid users = lrhorer, "Leslie A Rhorer"
	admin users = lrhorer, "Leslie A Rhorer"
	read only = No
	guest ok = No
	strict locking = No
	browseable = No

[Liza]
	path = /RAID/Personal_Folders/Liza/
	valid users = lgates
	admin users = lgates
	read only = No
	guest ok = No
	strict locking = No
	browseable = No

[Tiffany]
	path = /RAID/Personal_Folders/Tiffany/
	valid users = tgates
	admin users = tgates
	browseable = No
	strict locking = No

[Photos]
	path = /RAID/Photos/
	admin users = lrhorer
	read only = No
	guest ok = Yes

[DVD_Rip]
	path = /RAID/DVD
	valid users = lrhorer, root
	read only = No
	guest ok = Yes

[Thermostat]
	path = /usr/share/thermostat
	username = root
	valid users = lrhorer, root
	admin users = lrhorer, root
	read only = No
	guest ok = Yes

[html]
	path = /var/www
	valid users = lrhorer, root
	admin users = lrhorer, root
	read only = No
	guest ok = Yes
Note that I lock down the personal folders not only by setting guest = no and making the shares non-browseable, but I also set strict permissions on the Linux server itself. The personal directories and all the files and subdirectories in them are set for a permission of 700, 600, 500, 400, or 100 as the case may be, with the owner of every directory and file being the user themselves. By comparison, the /RAID/Photos, /RAID/Music, and /RAID/Recordings directories are set with permissions of 777, and most of the files are at least world readable and writable. Anyone can map those folders as drives on their PC.

Last edited by lrhorer : 06-04-2012 at 02:06 AM.
lrhorer is offline   Reply With Quote
Old 06-04-2012, 06:11 AM   #15
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by lrhorer View Post
As long as the firewall on the 655 is enabled and youhave your wireless segments properly secured, that is all that is needed
Perfect!
markmarz is offline   Reply With Quote
Old 06-04-2012, 06:12 AM   #16
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by lrhorer View Post
The short answer to that is: SAMBA.
Great! Thanks for the SAMBA intro, I'll get on it asap! Almost there!
markmarz is offline   Reply With Quote
Old 06-04-2012, 06:20 AM   #17
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Thumbs up

Quote:
Originally Posted by lrhorer View Post
Here is a good basic tutorial on setting up SAMBA.
And more than SAMBA; I hadn't found this site on my own, it's excellent for a Debian newbie. Heck, I didn't even know I was mispronouncing Debian in my head till I landed there.
markmarz is offline   Reply With Quote
Old 06-04-2012, 11:45 AM   #18
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
How were you pronouncing it? Duh-BEE'-un?

Of course, once SAMBA is set up, network clients more or less just think the file server is another Windows workstation (or a domain master) with network sharing enabled. If runing Windows, just map the drive as you would a drive from any other Windows machine. From another Linux machine, it is even easier, although for file sharing between Linux workstations, you might choose to employ NFS. NFS and SAMBA can easily be run on the same server, sharing the same directories on the LAN.

Last edited by lrhorer : 06-04-2012 at 11:51 AM.
lrhorer is offline   Reply With Quote
Old 06-05-2012, 05:45 AM   #19
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by lrhorer View Post
How were you pronouncing it? Duh-BEE'-un?
Dee' - bee - en

But Duh is good!

Quote:
Originally Posted by lrhorer View Post
Of course, once SAMBA is set up, network clients more or less just think the file server is another Windows workstation (or a domain master) with network sharing enabled.
Just curious .. is the transfer speed across SAMBA pretty much equivalent to FTP? Wondering if sending 10GB files between the server & my pc where I'll run videoredo is best with either? Anyway, I'll experiment on my own once I get it all set up. Very busy at work this week, may be awhile.
markmarz is offline   Reply With Quote
Old 06-05-2012, 06:58 AM   #20
unitron
Registered User
 
unitron's Avatar
 
Join Date: Apr 2006
Location: semi-coastal NC
Posts: 13,771
Quote:
Originally Posted by markmarz View Post
And more than SAMBA; I hadn't found this site on my own, it's excellent for a Debian newbie. Heck, I didn't even know I was mispronouncing Debian in my head till I landed there.
Is

Deb--eee--an

not correct?

I don't see how the spelling would suggest anything else.
__________________
(thisismysigfile)


"I am altering the deal. Pray I don't alter it any further."

Darth TiVo, 14 February, 2011
unitron is offline   Reply With Quote
Old 06-05-2012, 08:55 AM   #21
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by unitron View Post
Is

Deb--eee--an

not correct?

I don't see how the spelling would suggest anything else.
Quote:
1.6 How does one pronounce Debian and what does this word mean?

The project name is pronounced Deb'-ian, with a short e, and emphasis on the first syllable. This word is a contraction of the names of Debra and Ian Murdock, who founded the project. (Dictionaries seem to offer some ambiguity in the pronunciation of Ian (!), but Ian prefers ee'-an.)

I was taught that a vowel is long when followed by a consonant followed by another vowel. So a short 'e' is irregular in my book. But it's actually a contraction of Debra, so Deb.
markmarz is offline   Reply With Quote
Old 06-05-2012, 10:26 PM   #22
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by markmarz View Post
Just curious .. is the transfer speed across SAMBA pretty much equivalent to FTP?
It depends on the client, but yeah, mostly. The biggest time differential may be in file creation time, but with multi-Gig files, that is pretty much insignificant.

Quote:
Originally Posted by markmarz View Post
Wondering if sending 10GB files between the server & my pc where I'll run videoredo is best with either?
Sending via SAMBA will certainly take less of *YOUR* time, since you don't have to have an FTP step in between. It also means a lot less time overall, since you can edit the video file directly on the server.

What I did was create a series of directories on my Server-Main share (mapped to s: on the VideoRedo workstation) and a directory for videos (mapped to v: on the workstation). I have kmttg automatically run the videos through tivodecode to convert them .mpg and toss them to a directory on s:. Thhen kmttg automatically runs comskip on the videos to create a VRD project (.prj) file in the same directory. I edit the raw video files using the .prj files directly from the server, and then have the Batch Builder process the .prj files, implementing the cuts and recoding to h.264 in a .mp4 container, writing the temporary files to the local (to VRD) hard drive and writing the finished file to v: (or one of its directories if the program is a movie franchise installment or TV series episode). What I do then is push the processed files to one of the TiVos to check their integrity. After the files all transfer without error, I then run a routine that moves the metafiles over from the s: drive, does some extra processing on the metafiles - some interactive and some automatic, touches up the names of the video files, marks them as valid, and deletes the files on the s: drive.

Here is the script, if you or anyone else are interested:

Code:
#!/bin/bash
runDir=/usr/share/VideoScribe
logDir=$runDir/Logs
logFile=$logDir/verified.txt
recDir=/RAID/Recordings
metaDir=/RAID/Server-Main/Movies/TiVo_MPG
tmpDir=/tmp
monthList="Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"

filespec=$@

test=1
if [[ $1 == "--test" ]] || [[ $1 == "-t" ]]
then
	test=0
	filespec=$( echo $filespec | cut -d" " -f2- )
else
	echo "Usage note: --test or -t => test for verification only"
fi

force=1
if [[ $1 == "--force" ]] || [[ $1 == "-f" ]]
then
        force=0
	filespec=$( echo $filespec | cut -d" " -f2- )
else
        echo "Usage note: --force or -f => force processing of files"
fi

DefaultIFS=$IFS
abort=\<Abort\>
[[ -z $filespec ]] && echo No Args! && exit
#clear

find "$recDir" -iname "*$filespec*.mp[4g]" > "$tmpDir"/Verify1.tmp

if [[ $force == 1 ]] && [[ $test == 1 ]]
then
	while read fileName
	do
		grep -q "$fileName" "$logFile" || echo $fileName >> "$tmpDir"/Verify2.tmp
	done < "$tmpDir"/Verify1.tmp
	mv "$tmpDir"/Verify2.tmp "$tmpDir"/Verify1.tmp
fi

IFS=$'\n'
fileList=$( cat "$tmpDir"/Verify1.tmp )

select fileName in $fileList $abort
do
	IFS=$DefaultIFS
	[[ "$fileName" == $abort ]] && exit
	shortName=${fileName##*/}
	echo $shortName
	echo
	echo -n is this correct \(Y\)?
	tput cub 3
	read response < /dev/tty
	[[ -z $response || "$response" == "y" ]] && break
done < /dev/tty

if [[ $test == 0 ]]
then
	echo
	grep -q "$fileName" /usr/share/VideoScribe/Logs/verified.txt
	if [[ $? == 0 ]]
	then
		echo $shortName is already on the list of verified movies.
	else
		echo $shortName is not on the list of verified movies.
	fi
	echo
	exit 0
fi

dirName=${fileName%/*}
stubName=${shortName%.mp[4g]*}
PrjName=$stubName.VPrj
prjName=$stubName.Vprj
fileType=.mpg
echo $shortName | grep -q .mp4 && fileType=.mp4
echo

cd /usr/share/pyTivo/Unverified
[ -a "$shortName" ] && rm "$shortName"
[ -a "$shortName.txt" ] && rm "$shortName.txt"

cd "$metaDir"
[[ -a "$PrjName" ]] && rm "$PrjName"
[[ -a "$prjName" ]] && rm "$prjName"
[[ -a "$shortName" ]] && rm "$shortName"
[[ -a "$stubName.mpg" ]] && rm "$stubName.mpg"
[[ -a "$stubName.mpg.txt" ]] && mv "$stubName.mpg.txt" "$fileName.txt"
[[ -a "$stubName.edl" ]] && rm "$stubName.edl"
[[ -a "$stubName.txt" ]] && rm "$stubName.txt"
[[ -a "$stubName.log" ]] && rm "$stubName.log"
[[ -a "$stubName.logo.txt" ]] && rm "$stubName.logo.txt"

[[ $force == 0 ]] && grep -v "$fileName" $logFile > $logFile.tmp
[[ -a $logFile.tmp ]] && mv $logFile.tmp $logFile

grep -q "$fileName" "$logFile"
if [[ $? == 0 ]]
then
	echo $shortName is already on the list of verified movies.
	echo
	exit 0
else
	echo Renaming File...
	newName=$dirName/$( "$runDir/Rename" "$fileName" )
	echo "$newName" >> "$logFile"
	sort $logFile > "$logFile.tmp"
	mv "$logFile.tmp" "$logFile"
fi

cd "$dirName"
[[ "$newName" != "$fileName" ]] && [[ -a "$fileName.txt" ]] && mv "$fileName.txt" "$newName.txt"
[[ "$newName" != "$fileName" ]] && [[ -a "$fileName.jpg" ]] && mv "$fileName.jpg" "$newName.jpg"

if [[ $fileType == ".mp4" ]] && [[ -a "$stubName.mpg" ]]
then
	echo
	echo -n Remove "$stubName.mpg" \(N\)?
	tput cub 3
	read response < /dev/tty
	if [[ "$response" == "Y" || "$response" == "y" ]]
	then
		rm "$stubName.mpg"
		[[ -a "$stubName.mpg.txt" ]] && mv "$stubName.mpg.txt" "$stubName.mp4.txt"
		[[ -a "$stubName.mpg.jpg" ]] && mv "$stubName.mpg.jpg" "$stubName.mp4.jpg"
	fi
fi

touch "$newName.txt"
touch "$newName.jpg"
sync
"$runDir"/NameFix "$newName.txt"
"$runDir"/NoSpace "$newName.txt"

echo
echo Updating Links...

LinkName=`echo ${newName##*/RAID/Recordings/} | sed 's/\//-/'`       # Get the root file name after the Recordings dirname for the video link
LinkMeta="$LinkName".txt	# Set the name of the Metafile link
LinkThumb="$LinkName".jpg	# Set the name of the Thumbnail link

# Set up the Genre name array
Genre="Action..... Adventure.. Animated... Biography.. Classic.... Comedy..... Crime...... Documentary Drama...... Family..... Fantasy.... Film_Noir.. Holiday.... Horror..... Musical.... Mystery.... Nature..... Romance.... Science.... SciFi..... Series..... Thriller... Tragedy.... War........ Western.... Done"

echo
echo ${newName##*/}
echo
grep vProgramGenre "$newName.txt"
grep vSeriesGenre "$newName.txt"
echo

Gselect=""
x=0
select Gselect in $Genre;	# Assign Genres to the video
do
	if [ "$Gselect" == "Done" ]
	then
		break;
	else
		Gselect=`echo $Gselect | tr -d '.'`
		cd /usr/share/pyTivo/pyshares/$Gselect
		[[ -h "$LinkName" ]] || ln -s "$newName" "$LinkName"
		[[ -h "$LinkMeta" ]] || ln -s "$newname.txt" "$LinkMeta"
		[[ -h "$LinkThumb" ]] || ln -s "$newName.jpg" "$LinkThumb"
		x=$(( $x + 1 ))
		metaFields[$x]=$Gselect
	fi
done < /dev/tty
grep -v vProgramGenre "$newName.txt" | grep -v vSeriesGenre > "$newName.txt.tmp"

for (( y=1; y<=$x; y++ ))
do
	echo ${metaFields[$y]}
	echo "vProgramGenre : ${metaFields[$y]}" >> "$newName.txt.tmp"
done
mv "$newName.txt.tmp" "$newName.txt"

vName=${newName##*/}.txt
datName=${newName##*Recorded }
vName=${vName% (Recorded*}
x=1
for mon in $monthList
do
	[[ $mon == ${datName:4:3} ]] && monNum=$(( 22 - $x ))
	x=$(( $x + 1 ))
done
if [[ ${datName:8:1} == "0" ]]
then
	dayNum=$(( 41 - ${datName:9:1} ))
else
	daynum=$(( 41 - ${datName:8:2} ))
fi
yearNum=$(( 3200 - ${datName:12:4} ))
grep -q "$yearNum" "$newName.txt" || echo "recordDate : $yearNum$monNum$dayNum" >> "$newName.txt"
grep -iq "isEpisodic : true" "$newName.txt" || grep -iq "firstAlpha : " "$newName.txt"
if [[ $? != 0 ]]
then
	vTitle=$( grep title "$newName.txt" )
	firstAlpha=${vTitle:8:1}
	[[ $firstAlpha == [*\(\?] ]] && firstAlpha=${vTitle:9:1}
	[[ ${vTitle:8:4} == "The " ]] && firstAlpha=${vTitle:12:1}
	[[ ${vTitle:8:2} == "A " ]] && firstAlpha=${vTitle:10:1}
	[[ $firstAlpha == [0-9] ]] && firstAlpha="0 - 9"
	echo "firstAlpha : $firstAlpha" >> "$newName.txt"
fi

grep \| "$newName.txt" | grep -q ":"
if [[ $? == 0 ]]
then
	echo -n > "$newName.txt.tmp"
	while read metaText
	do
		echo $metaText | grep \| | grep -v "description" | grep -q ":"
		if [[ $? == 0 ]]
		then
			colonPos=$( expr index "$metaText" ":" )
			pipePos=$( expr index "$metaText" "\|" )
			nameLength1=$(( ${#metaText} - $pipePos -1 ))
			[[ $nameLength1 -lt 0 ]] && nameLength1=0
			nameLength2=$(( $pipePos - $colonPos - 1 ))
			[[ $nameLength2 -lt 0 ]] && nameLength2=0
			echo -e "${metaText: 0: $colonPos} ${metaText: $pipePos: $nameLength1}${metaText: $colonPos: $nameLength2}\r" >> "$newName.txt.tmp"
		else 
			echo $metaText >> "$newName.txt.tmp"
		fi
	done < "$newName.txt"
	mv "$newName.txt.tmp" "$newName.txt"
fi

grep -q "callsign : " "$newName.txt"
if [[ $? != 0 ]]
then
	echo -n > "$newName.txt.tmp"
	x=0
	while read line
	do
		echo $line >> "$newName.txt.tmp"
		x=$(( x + 1 ))
		if [[ $x == 5 ]]
		then
			callSign=${newName##*, }
			echo callsign : ${callSign%%)*} >> "$NewName.txt.tmp"
		fi
	done < "$newName.txt"
	mv "$newName.txt.tmp" "$newName.txt"
fi

echo Done!
I apologize just a bit for the lack of comments. This is pretty short and I wrote it for my own use.
lrhorer is offline   Reply With Quote
Old 06-09-2012, 06:14 AM   #23
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Question

Quote:
Originally Posted by wmcbrine View Post
On the rare occasions when I want to run an X app remotely, I just do an "ssh -X" and start the app from the command line.

Leaving an X environment running full-time on a headless server can be a waste of server resources.
Okay, I've got X-Ming working on my pc and have been able to login to my tivo server at a static ip with putty and then kick off a gui (Leafpad) as a test, thanks to lrhorer.

But I'm still a little confused about the idea of not running X windows 'all the time' on my tivo server. I think you're saying that if I run "ssh -X" when logged in with putty that I can somehow kick off a gui on the tivo server. I tried that and got a syntax error.

Code:
mark@tivo-server-backup:~$ ssh -X
usage: ssh [-1246AaCfgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec]
           [-D [bind_address:]port] [-e escape_char] [-F configfile]
           [-I pkcs11] [-i identity_file]
           [-L [bind_address:]port:host:hostport]
           [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port]
           [-R [bind_address:]port:host:hostport] [-S ctl_path]
           [-W host:port] [-w local_tun[:remote_tun]]
           [user@]hostname [command]
mark@tivo-server-backup:~$
Even so, if this did work, what would actually be happening? Would I be kicking off X on the tivo server, and if yes, how would that enable me to see the gui on my pc if I'm not running an X server there? Or maybe you're saying I do need to run X-Ming on my pc, but no X server needs to run on the tivo server all the time, instead I can kick it off adhoc using ssh -X (or whatever actually works)?

And if I wanted to not run X all the time on the tivo server, how exactly do I disable or prevent it from running and yet be available adhoc?

This is just for my knowledge, I haven't come down on either side of whether or not to run X all the time on a headless server because I don't know enough to form an intelligent position.
markmarz is offline   Reply With Quote
Old 06-09-2012, 09:32 AM   #24
wmcbrine
Resistance Useless
 
wmcbrine's Avatar
 
Join Date: Aug 2003
Posts: 9,156
Quote:
Originally Posted by markmarz View Post
I think you're saying that if I run "ssh -X" when logged in with putty that I can somehow kick off a gui on the tivo server.
No. You'd be using ssh to log in. Not really relevant if you want to use PuTTY. "-X" is just an ssh option that makes it easier to start X apps from the login session. I don't suppose PuTTY has an equivalent, but really, all you have to do is export the DISPLAY variable appropriately, and make sure the client isn't blocked from connecting to your X server by a firewall.

Quote:
Or maybe you're saying I do need to run X-Ming on my pc
Yes.

Quote:
but no X server needs to run on the tivo server all the time, instead I can kick it off adhoc using ssh -X (or whatever actually works)?
No X server needs to run on the tivo server ever. You can run X apps (clients) one at a time, and all they need is an X server on any machine to connect to. The one on your PC will do fine.

Quote:
And if I wanted to not run X all the time on the tivo server, how exactly do I disable or prevent it from running and yet be available adhoc?
Usually it's a matter of changing the runlevel. This varies by distro. I don't know offhand for Debian.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
wmcbrine is offline   Reply With Quote
Old 06-09-2012, 11:29 AM   #25
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
One thing that may be confusing you (it confuses a lot of people) is that the X client / server relationship is a little backwards from what most people expect. Most "servers" run on a dedicated machine, often one without a console or mouse attached. PyTivo, for example, is perfectly happy running on a headless machine. Most "clients" run on potentially multiple machines with users sitting at them. The X server (in your case Xming) is different, however. The X server runs on user machines, while the X clients run on a possibly console-less machine, but the application puts up its display on the machines running an X server.

It doesn't particularly matter from what machine the client is spawned nor on which machine the client is displayed. One could, for example log in to machine B from machine A using telnet, set the $DISPLAY variable to point to machine C, and spawn an X app on machine B. The app would then pop up on machine C, assuming it is available and running an X server. Neither machine A nor machine B need to be running an X server in this scenario, nor do they need to be running a GUI. Either or both can be doing so, but it is not required for this example. One could also run a cron job on an X client machine to spawn a client at regular intervals. The app would then pop up on the targeted machines every time cron spawned it.
lrhorer is offline   Reply With Quote
Old 06-09-2012, 12:22 PM   #26
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by wmcbrine View Post
No. You'd be using ssh to log in. Not really relevant if you want to use PuTTY. "-X" is just an ssh option that makes it easier to start X apps from the login session. I don't suppose PuTTY has an equivalent, but really, all you have to do is export the DISPLAY variable appropriately, and make sure the client isn't blocked from connecting to your X server by a firewall.
Right you are, of course. IOW, any means of running the X app on the client machine can be employed. It doesn't even have to be something remote to the client, but it can be. One can use rsh, ssh, telnet, PuTTY, a canned utility built in to the X server, whatever. Anything that can export the $DISPLAY variable and then run the app will work.

Quote:
Originally Posted by wmcbrine View Post
No X server needs to run on the tivo server ever.
He may not even have one installed. The Debian installer does not install an X server by default. As you pointed out, he also does not necessarily need to run a display manager, unless he wants to run the GUI on the TiVo server, either from the console or from XDMCP (or other remote desktop software such as VNC).

Quote:
Originally Posted by wmcbrine View Post
Usually it's a matter of changing the runlevel. This varies by distro. I don't know offhand for Debian.
Well, that is handled by the init scripts and the display manager. Both GDM and KDM are enabled for all runlevels other than 0, 1, and 6. I'm not certain about the display manager Mark is using, but it is likely the same. Dropping to runlevel 1 (single user) would disable the GUI, but it also disables networking. Runlevels 0 and 6 shut down the machine.

One way to disable the GUI completely is to simply disable the display manager. This can be accomplished by putting an `exit 0` statement at the top of the init script, or by changing the runlevel parameters in the init script and updating using the update-rc.d command. I don't recommend any of these methods, however, especially if he might ever want to use the GUI.

Now you are perfectly correct that running a GUI on a server is not considered best practice. You are also perfectly correct that having a GUI display up full time is a waste of at least some resources on the server. It is not such a terrible waste of resources, however, to leave the display manager daemon (which is tiny and uses almost no resources) running. One can also disable the GUI for the console. OTOH, even the greeter (not the full GUI) running on the console only really uses resources on the video display, and that is pretty much irrelevant to the operation of the server as a whole.

Really, I would say as long as he does not log in on the console and does not leave an XDMCP session up all the time, he is mostly just as well not to worry about it. If he really, really wants to minimize resources, though, as I say he can disable his DM for the console but leave it active for remote logins.
lrhorer is offline   Reply With Quote
Old 06-09-2012, 12:44 PM   #27
lrhorer
Registered User
 
Join Date: Aug 2003
Location: San Antonio, Texas, USA
Posts: 6,893
Quote:
Originally Posted by markmarz View Post
Okay, I've got X-Ming working on my pc and have been able to login to my tivo server at a static ip with putty and then kick off a gui (Leafpad) as a test, thanks to lrhorer.
That is not a GUI. Leafpad is an X client displayed in an X-window on your X server. The GUI in this case is Windows, and the only GUI running on the server is (possibly) whatever is on the server's console, probably just the greeter.

Between this and William's and my other responses, do you get the picture? This is a screenshot of KDE, the GUI I am using on my Linux systems:



Last edited by lrhorer : 06-09-2012 at 01:04 PM.
lrhorer is offline   Reply With Quote
Old 06-09-2012, 01:45 PM   #28
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by wmcbrine View Post
.. but really, all you have to do is export the DISPLAY variable appropriately, and make sure the client isn't blocked from connecting to your X server by a firewall.
Okay, I'll dig further and figure out what that means.

In the meantime, thanks!

Quote:
No X server needs to run on the tivo server ever. You can run X apps (clients) one at a time, and all they need is an X server on any machine to connect to. The one on your PC will do fine.
Gotcha, thanks!
markmarz is offline   Reply With Quote
Old 06-09-2012, 01:50 PM   #29
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by lrhorer View Post
That is not a GUI. Leafpad is an X client displayed in an X-window on your X server. The GUI in this case is Windows, and the only GUI running on the server is (possibly) whatever is on the server's console, probably just the greeter.
I get the picture (literally) . I was speaking loosely when I said Leafpad was a gui; of course it isn't itself a gui like windows. Then again, it's not a command line text editor; it has a graphical user interface.
markmarz is offline   Reply With Quote
Old 06-09-2012, 01:53 PM   #30
markmarz
Registered User
 
Join Date: Feb 2002
Location: Chicago, IL
Posts: 80
Quote:
Originally Posted by lrhorer View Post
One thing that may be confusing you (it confuses a lot of people) is that the X client / server relationship is a little backwards from what most people expect.
You hit the nail on the head, thanks! And thanks for the $DISPLAY variable explanation.
markmarz is offline   Reply With Quote
Reply
Forum Jump




Thread Tools


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Advertisements

TiVo Community
Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
vBulletin Skins by: Relivo Media

(C) 2013 Magenium Solutions - All Rights Reserved. No information may be posted elsewhere without written permission.
TiVoŽ is a registered trademark of TiVo Inc. This site is not owned or operated by TiVo Inc.
All times are GMT -5. The time now is 05:05 AM.
OUR NETWORK: MyOpenRouter | TechLore | SansaCommunity | RoboCommunity | MediaSmart Home | Explore3DTV | Dijit Community | DVR Playground |