1	The Last Mile Integrating UniVerse with Free Software Tools or I wrote a web server last Sunday… Ross Morrissey Spectrum 2004
2	The Last Mile ODBC? Plain Text HTTP Web Browsers Web Servers
3	Getting Data from Universe To the Web/Intranet Asynchronous Approach UniVerse Runs Before UniVerse Runs After Synchronous Approach Built-in Tools Named Pipes Sockets
4
5	Making &HOLD& available Symbolic Links httpd.conf Aliases
6	Symbolic Links Hard Links
7	Symbolic Linking cd /usr/local/etc/apache/htdocs ln -s /uvhome/\&HOLD\& reports
8	Symbolic Links Advantages Easy to set up See everything by default Disadvantages Easy to set up See everything by default Doesn't do Windows
9	httpd.conf Aliases Minimal setup: Add this line to apache/conf/httpd.conf: Alias /rossh/ /u1/home/ross/ROSS/&HOLD&/ Restart Apache with a command like: kill -HUP `cat /usr/local/apache/logs/httpd.pid`
10	httpd.conf Directives <Directory ~ "////\&HOLD\&"> Options Indexes MultiViews AllowOverride None Order allow,deny Allow from all AuthType Basic AuthName "Password Required“ AuthUserFile /www/passwords/holduser AuthGroupFile /www/passwords/holdgroup Require Group admins </Directory>
11	httpd.conf Aliases Disadvantages Harder to set up See nothing by default Advantages Harder to set up See nothing by default Does do Windows
12	Publishing to the Intranet &HOLD& Files Raw Text HTML SAVE-LIST Virtual Keys tabbed or .CSV
13	&HOLD& Files - Raw Text SETPTR Option 3 - &HOLD& AS .txt or .text
14	&HOLD& Files - Raw Text SETPTR 0,,,0,0,3,AS listf.txt,NOHEAD LIST VOC LPTR
15
16
17	Spoofing URLs DICT DIP ROLL.LINK 0001: I 0002: '<a href= http://photomail.filmworks.com /pmlogin.asp?P1=' : CUST.NO : '&P2=' : ROLL.NUM : '&P3=0">' : ROLL.NUM : '</A>' 0003: 0004: Original ]Roll 0005: 99L 0006: S
18
19	Populating a Form ED DICT DIP RE.RIP 0001: I 0002: ROLL.LINK :'': '<A HREF="': 'http://filmworks.filmworks.com/cgi/control.cgi?Application=RE.RIP&HIPId=':HIP.ID:'&Effect=':EFFECT :'&OrderId=':SO.ID: '&OriginalPD=' :ROLL.NUM: '&Frame=' :IMAGE:'&Process=D' :SIZE+2:'X':EFFECT: '&Quantity=' :QUANTITY: &BizCode=0':1:'&Button=Load':'"> [ReRIP]</A>' 0003: 0004: Original ]Roll 0005: 309L 0006: S
20
21	"Freeze Frames" Problem: Report Header only appears at the top of each “page” Solution: Put the report in a “frame”
22	"%more dps.html" %more dps.html <HTML> <HEAD><TITLE>DPS Report</TITLE></HEAD> <FRAMESET ROWS="65,*"> <FRAME SRC="dps.headers.html" NAME=foo> <FRAME SRC="dps.frame.html" NAME=bar> <NOFRAMES><BODY>...</BODY></NOFRAMES> </FRAMESET> </HTML>
23	"DPS.UNSHIPPED" DPS.UNSHIPPED 0001: PA 0002: SETPTR 0,999,9999,0,0,3,AS dps.to.ship.html, NOHEAD 0003: GET-LIST PD.TO.SHIP 0004: SORT PD WITH STATUS NE "X" BY-DSND SO.ID BY @ID POST.DATE DATE CI.ID CI.NAME LTYPE LQTY FRAMES SO.ID DPS.BATCH DRIVE STATUS DPS.REPORT HEADING "<html><head><title>DPS Unshipped at 'T'</title></head><body><H3>Digital Unshipped at 'T'</H3><PRE>" LPTR ID-SUPP 0005: SH -c "cd /sfw/SFW/DOS ; cp dps.to.ship.html dps.frame.html" 0006: SH -c "/sfw_uv/scripts/create.nondps.report.sh"
24
25	Filtered Reports Problem: We need a SELECT list subset of an expensive report Solution: Filter the expensive report using grep
26	Filtered Reports REPORT.PARSE 0001: PA 0002: SH -c 'grep "<<Enter String>>" /usr/local/etc/apache/htdocs/reports/dps.frame.html \| cut -c 78-85 > /uvhome/\&SAVEDLISTS\&/'<<S2>> 0003: GET-LIST <<S2>> >REPORT.PARSE TEST Enter String=Non 98 record(s) selected to SELECT list #0. >>
27	Generating Histograms In the following example, the I-descriptor GRAPH.LESSONS defines an I-type expression that uses the STR function: GRAPH.LESSONS 0001 I 0002 STR('*',LESSONS) 0003 0004 0005 50L 0006 S
28	Generating Histograms >LIST SUN.SPORT LESSONS GRAPH.LESSONS GRAPH.LESSONS.......... 85-1000 4 ** 85-1006 4 85-1008 8 **** 85-1016 14 ********** 85-1020 9 ***** 85-1015 3 * 85-1005 5 *** 85-1007 10 ******** 8 records listed.
29	"BASH-2.01$" BASH-2.01$ more blue.box.htm <html> <img src="blue.gif" > <img src="blue.gif" height=10 width=10> <img src="blue.gif" height=20 width=20> <img src="blue.gif" height=40 width=40> <img src="blue.gif" height=80 width=80> <img src="blue.gif" height=160 width=160> </html>
30
31	Generating Histograms The I-descriptor GIF.BAR defines an I-type expression that abuses the STR function: 001: I 002: DCOUNT(@RECORD,@AM);IF @1 LT 1000 THEN '<IMG SRC="../green.gif" WIDTH="' : @1/2 : '" HEIGHT="10" ALT = "' : @1 : '">' ELSE '<IMG SRC="../red.gif" WIDTH="' : @1/20 : '" HEIGHT="10" ALT = "' : @1 : '">' 004: LINES 005: 50L 006: S
32	Generating Histograms VOC BP.REPORT 0001: PA 0002: TERM 9999,9999 0003: SORT <<I2,FILE>> LINES GIF.BAR HEADING "<HTML><HEAD><TITLE>Basic Program Sizes for <<I2,FILE>> at 'T'</TITLE></HEAD>'L'<BODY> <H3>Basic Program Sizes for <<I2,FILE>> at 'T'</H3><PRE>“
33
34
35
36	SAVE-LIST – "Virtual Keys" Tabbed grep, wc, cut, sort, awk, perl import into SQL Server, Excel, Access .CSV comma separated values Associated with Excel Opens Excel session from Browser
37	SAVE-LIST - Virtual Keys - Tabbed ED MKT.PL JLSTATS 0001: PA 0002: GET-LIST ML JLDSANAL2 0003: SSELECT CRX 0004: SELECT CRX COUPON.NBR 0005: SELECT MCP SAVING STATS.DS 0006: SAVE-LIST MKTSHARE JLSFW.TXT
38	SAVE-LIST - Virtual Keys - Tabbed Source ED DICT MCP STATS.DS 0001: I 0002: COUPON.CODE'L#6':CHAR(9):@ID:CHAR(9): INIT.ORD'R#9':CHAR(9):TOT.ROLLS'R#9': CHAR(9):NEW.CUSTS'R#9':CHAR(9): REVENUE.MR2'R#12':CHAR(9):ART.NBR 0003: 0004: stats 0005: 80L 0006: S
39	SAVE-LIST - Virtual Keys - Tabbed Example
40	SAVE-LIST - Virtual Keys - .CSV ED MKT.PL JLSTATS.CSV 0001: PA 0002: GET-LIST ML JLDSANAL2 0003: SSELECT CRX 0004: SELECT CRX COUPON.NBR 0005: SELECT MCP STATS.DS.CSV 0006: SAVE-LIST MKTSHARE JLSFW.CSV
41	SAVE-LIST - Virtual Keys - .CSV ED DICT MCP STATS.DS.CSV 0001: I 0002: '"':COUPON.CODE'L#6':'","':@ID:'","': INIT.ORD'R#9':'","':TOT.ROLLS'R#9':'","': NEW.CUSTS'R#9':'","':REVENUE.MR2'R#12': '","':ART.NBR:'" 0003: 0004: stats 0005: 80L 0006: S
42
43	Automating Report Production crontab setup shell script setup UniVerse PAragraph setup
44	crontab setup $ crontab –l \| grep hourly 0 * * * * /sfw_uv/scripts/uv.hourly < /dev/null 2>&1>/dev/null
45	shell script setup $ cd /sfw_uv/scripts/ $ cat uv.hourly #/bin/sh cd /sfw/SFW /usr/opt/uv/bin/uv HOURLY & cd /sfw/SFW.B /usr/opt/uv/bin/uv HOURLY &
46	UniVerse PAragraph setup ED VOC HOURLY 001 PA These commands are invoked by cron every hour 002 HIP.INCOMING 003 PHANTOM HIP.ORDER 004 PHANTOM HIP.CS
47	Dynamic Report Production "We need a new report just like the other one for this new product" Create Reports Dynamically Create Report Menus Dynamically
48
49	Report Command DICT CPC PW.REP1.CMD 0001: I 0002: “PW.REP1 " : @ID : " " : DOWNCASE(@ID) : ".report1.html " : PARTNER.NAME
50	Report Command SORT CPC PW.REP1.CMD ID-SUPP PW.REP1 P1 p1.report1.html Partner One PW.REP1 P2 p1.report1.html Partner Two PW.REP1 P4 p1.report1.html Partner Four 3 records listed..
51	Report Routine PW.REP1 P1 p1.report1.html Partner One PA SETPTR 0,999,9999,0,0,3,AS <<C2>>,BRIEF,NOHEAD,INFORM SORT TRANSACTION WITH OUTSTANDING EQ “1” AND WITH PARTNER EQ “<<C1>>” PW1.PH HEADING “<HTML><HEAD><TITLE> <<C1>> - <<C3>> <<C4>> <<C5>> </HEAD><BODY><PRE><H3> <<C1>> - <<C3>> <<C4>> <<C5>> </H3><PRE>” LPTR
52	Menu Line DICT CPC PW.MENU 0001: I 0002: '<A HREF="' : DOWNCASE(@ID) : '.report1.html">' : @ID : ' - ' : PARTNER.NAME: ' Report</A>'
53	"SORT CPC PKI.MENU ID-SUPP" SORT CPC PKI.MENU ID-SUPP <A HREF=“p1.report1.html">P1 – Partner One Report</A> <A HREF=“p2.report1.html">P2 – Partner Two Report</A> <A HREF=“p4.report1.html">P4 – Partner Four Report</A>
54	"PKI.REPORTS" PKI.REPORTS 0001: PA 0002: SELECT HIP.SOURCE WITH BIZ.CODE EQ "06" SAVING UNIQUE CPC.ID 0003: SELECT CPC SAVING PW.REP1.CMD 0004: RUN.READNEXT
55	"BP RUN.READNEXT" BP RUN.READNEXT LOOP READNEXT COMMAND ELSE EXIT EXECUTE COMMAND CAPTURING X REPEAT
56	"CREATE.PW1.MENU" CREATE.PW1.MENU 0001: PA 0002: SETPTR 0,999,9999,0,0,3,AS pw1.menu.html,BRIEF,NOHEAD,INFORM 0003: SORT CPC PKI.MENU HEADING "<html><head><title>Partner Reports</title></head><body><H3> Partner Reports</H3><PRE>" LPTR REQUIRE.SELECT ID-SUPP
57
58
59	Keeping Report History >ED VOC DPS.STATUS.REPORT 0001: PA 0002: SETPTR 0,999,9999,0,0,3,AS dps.status.temp,BRIEF,NOHEAD,INFORM,COPIES 1 0003: RUN CGI.BP DPS.STATUS.REPORT LPTR 0004: SH -c "cd /usr/local/etc/apache/htdocs/cgi-bin/\&HOLD\& ; cp dps.status.temp dps.status.html" 0005: SH -c "cd /usr/local/etc/apache/htdocs/cgi-bin/\&HOLD\& ; cp dps.status.temp dps.status/`date '+%y.%m.%d-%H:00.html'`"
60
61	Running Scripts The Environment Enabling Scripts Static Scripts Dynamic Scripts - get
62	Editing Apache httpd.conf
63	Simple HTTP-aware Unix Scripts cd /home/ross/uvhome/ More test.cgi #!/bin/sh echo Content-type: text/html echo echo Hello World
64	"Make it readable and executable..." Make it readable and executable for everybody!!! chmod a+rx test.cgi test directly from Unix… ./test.cgi Content-type: text/html Hello World
65
66	Passing Commands to UniVerse from Unix /usr/opt/uv/bin/uv TIME 15:10:39 15 JUL 2003
67	"more test.cgi" more test.cgi #!/bin/sh echo Content-type: text/html echo /usr/opt/uv/bin/uv "$*" ./test.cgi TIME Content-type: text/html 15:20:51 15 JUL 2003
68
69
70	"more pre.cgi" more pre.cgi #!/bin/sh echo Content-type: text/html echo echo "<html><head><title>$</title></head>" echo "<body><pre>" /usr/opt/uv/bin/uv "$" echo “</pre></body></html>" chmod +x pre.cgi
71
72	Passing Arguments to UniVerse using FORM Quick Forms Background Two components in web forms form tags interpreted by the web browser CGI (common gateway interface) scripts that handle the data on the web server end.
73	Google - front end
74	"<form action="/search" <form action="/search" name=f> <table cellspacing=0 cellpadding=0> <tr> <td width=75> </td> <td align=center> <input maxLength=256 size=55 name=q value=""> <br> <input type=submit value="Google Search" name=btnG> <input type=submit value="I'm Feeling Lucky" name=btnI> </td>
75	"<td valign=top nowrap>" <td valign=top nowrap> <font size=-2>   <a href=/advanced_search> Advanced Search</a> <br>   <a href=/preferences>Preferences</a> <br>   <a href=/language_tools>Language Tools</a> </font></td></tr></table> </form>
76	Google fragment
77	Google - front end
78	"<form action=“/search" <form action=“/search" name=f> http://www.google.com/search? q=smarty&btnG=Google+Search
79	Name/Value Pairs <input maxLength=256 size=55 name=q value=""> http://www.google.com/search? q=smarty&btnG=Google+Search Shell variable $QUERY_STRING contains: q=smarty&btnG=Google+Search
80	Name/Value Pairs <input type=submit value="Google Search" name=btnG> http://www.google.com/search? q=smarty&btnG=Google+Search Shell variable $QUERY_STRING contains: q=smarty&btnG=Google+Search
81	Google - back end more search #!/bin/sh echo Content-type: text/html echo /usr/opt/uv/bin/uv GET.SEARCH
82	"Shell variable $QUERY_STRING contains" Shell variable $QUERY_STRING contains: q=smarty&btnG=Google+Search
83	"GOO.BP GET.SEARCH" GOO.BP GET.SEARCH EXECUTE "SH -c 'echo $QUERY_STRING'" CAPTURING PARAMETERS CONVERT "&=+" TO @AM:@VM:" " IN PARAMETERS NUMBER.OF.PAIRS = DCOUNT(PARAMETERS,@AM) NAME = ""; VALUE = "" FOR I = 1 TO NUMBER.OF.PAIRS NAME<I> = PARAMETERS<I,1> VALUE<I> = PARAMETERS<I,2> NEXT I
84	"LOCATE "q" IN NAME..." LOCATE "q" IN NAME SETTING POS THEN QUERY = VALUE<POS> ELSE QUERY = "" LOCATE "btnG" in NAME SETTING POS THEN CALL GOOGLE.SEARCH(QUERY,RESULT) ELSE CALL LUCKY.SEARCH(QUERY,RESULT) PRINT RESULT RETURN
85	"What did we learn from..." What did we learn from Google? The Front End Form tag structure Input tag structure Using Buttons The Back End Parsing arguments Returning results
86
87
88	"Firing up Universe" Firing up Universe BP UVREAD.CLONE GET(ARG.) FILENAME ELSE STOP OPEN FILENAME ELSE STOP ITEM = "" GET(ARG.) KEY READ ITEM FROM KEY THEN PRINT KEY: NUMBER.OF.LINES = DCOUNT(ITEM,@AM) FOR I = 1 TO NUMBER.OF.LINES PRINT CHAR(9) : ITEM<I> NEXT I END
89	"BASH-2.01$" BASH-2.01$ /usr/opt/uv/bin/UVread "RAWTEST 3" 3 ATTR1 ATTR2 bash-2.01$ /usr/opt/uv/bin/uv "UVREAD.CLONE RAWTEST 3" 3 ATTR1 ATTR2
90	"bash-2.01$" bash-2.01$ sort -u uvreadbench.sh /usr/opt/uv/bin/uv "UVREAD.CLONE RAWTEST 3" > /dev/null bash-2.01$ wc -l uvreadbench.sh 100 uvreadbench.sh bash-2.01$ time ./uvreadbench.sh real 0m24.148s user 0m5.809s sys 0m18.896s
91	"Firing up Universe (again" Firing up Universe (again) UVread UVwrite Uvdelete /usr/opt/uv/bin/UVread RAWTEST 3 ATTR1 ATTR2
92	"BASH-2.01$" BASH-2.01$ sort -u uvreadbench.sh /usr/opt/uv/bin/UVread RAWTEST 3 > /dev/null BASH-2.01$ wc -l uvreadbench.sh 100 uvreadbench.sh BASH-2.01$ time ./uvreadbench.sh real 0m12.281s user 0m2.156s sys 0m9.845s
93	"UniVerse as Server - Named..." UniVerse as Server - Named Pipes /etc/mknod name p BASH-2.01$ ls -l pipe* prw-rw-r-- 0 Feb 15 15:20 pipe prw-rw-r-- 0 Feb 16 10:44 pipe1 prw-rw-r-- 0 Feb 16 10:44 pipe2
94
95	"bash-2.01$" bash-2.01$ sort -u pipebench.sh echo "RAWTEST 3" > pipe1 bash-2.01$ wc -l pipebench.sh 100 pipebench.sh bash-2.01$ time ./pipebench.sh real 0m0.430s user 0m0.169s sys 0m0.256s
96	"UniVerse as Server - Sockets" UniVerse as Server - Sockets bash-2.01$ lynx -source http://develop:4002/READ+RAWTEST+3 3 ATTR1 ATTR2
97
98
99
100	"bash-2.01$" bash-2.01$ time /usr/opt/uv/bin/uv/ SOCKET.SERVER Break: Option (A,C,L,Q,D,?) =Q real 2m57.606s user 0m0.440s sys 0m1.396s
101
102
103
104
105
106
107