|
|
MAY has been an eventful month . The first "Turn-on-and-use" K.CoSy NoteComputer
was successfully set up and delivered . It is , in fact , the first ready to use CoSy NoteComputer delivered since my being diverted by family matters in 1987 .
Only when prepared , even customized , ready to "turn-on-and-use" - does the powerful simplicity of KoSy become apparent . For many potential users of KoSy , like my old friend Raj , learning Windows to a truly useable level , is as large a task as learning the basic structures and operations of K.CoSy well enough to do what they need to do . And in K.CoSy , the rest is there , all open and accessible and coherent - a lifetime's worth .
Contact me if you have an interest one of these "Ultimate NoteKomputers"
based on top of the line Sony notebooks .
Jack Andrews , on the K language mail group , asked :
> i'd be intrigued to know what sort of people do use CoSy and what they
> use it for.
Raj got it partly for friendship , partly because he knows if we are going to do any business together , it will be run in K.CoSy .
I think the score of subscribers who have access to K.CoSy around the world thru the web , mainly , at this point , have used it just for exploration . It has been available just 10 months , and was minimal to say the least -- which is why I called it "extreme" and charter subscriptions have been so cheap . With the polishing inherent in creating the first turn-on-and-use machine , the next release will much smoother to use . I've uploaded the upgraded initial .CoSy.text variable .
One use of K.CoSy is that , , given the paucity of books on K , K.CoSy provides an interactive introduction to K and hundreds of examples of useful words written in pretty decent K .
But most fundamentally , I'm User 0 . 
That's why I like the analogy of Colin Chapman's Lotus super 7 like this one I happened to spot in London after the Lancaster APL96 conference .
Chapman built cars essentially to suit himself . An aestheic of lean performance dominated all his creations .
Likewise , CoSy is what I've created to maximize my own efficiency dealing with the business of everyday life . An awful lot of this boils down to keeping records , both logs and accounts , of a number of miscellaneous interests and responsibilities .
I have a head that works ( to the extent that it does at all ) at the APL - J - K level . Anything less is wasting my time , and drives me nuts . As I encapsulate more and more of the tasks I have to do , as the accounting cycles roll along , into vocabulary in K.CoSy , I free more and more of my time and energy for addition interests and responsibilities . An example of this sort of thing is the conversion of Schwab transaction histories below .
Since most of these activities are common to any taxpayer , or company officer , I anticipate a market , perhaps thin but global , among that level of individual , particularly when custom interfaced to the important information systems of their businesses .
And , of course , because it is an open linguistic system , I see it being a platform in which others can add their own expertises to present custom machines for their own clients and associates .
|
I Look forward to meeting some of youall during the SIA Tek Expo when Kx will be in town .
I plan to have an open house down here at Peck Slip , one of the evenings of the conference |
|
#HTMLExtract
In response to a question ( below ) on the Kdb mail group , I have annotated
the process of converting an html table to a K "DictionaryTable" or data base .
/(/ HTML Table Extraction \/ ==================== \/ ==================== \/
/)/
/ Used `open phrase in f8 command window to get file name
.r : 0: "C:/CoSy/ArmstrongFamily/JhaJr/2002/schwab020528.htm"
/ looked at whole html using an editor ( CuteHTML ) and found the following delimiters :
Q : ("<!-- Start Transaction table -->" ; "<!-- End Transaction table -->")
.r : VMn @ * tokcut[ Q 1 ] @ *| tokcut[ Q 0 ] MVn .r / extract transaction table
/ tokcut/[ MVn .r ; Q ] />/ (1;"nonce") / I'd like to see an expression that works .
: .r : DAE @ htmlcomdel' .r / delete comments and empty rows
"C:/CoSy/ArmstrongFamily/JhaJr/2002/schwab020528tbl.htm" 0: .r / saved interim
.r : 0: "C:/CoSy/ArmstrongFamily/JhaJr/2002/schwab020528tbl.htm" / resume
html2tab..toks / : { lower PRTBF[ DLB x ; ">" ] }'' tab2html..toks
(("<td"
"</td")
("<tr"
"</tr")
("<table"
"</table"))
html2tab..toks..h />/ " tokens for converting html tables " / not used here
/ starting a line with : causes the result to be displayed in `r
: Q : DAE htmlcut[ "tr" ; lower MVn .r ] / split rows
: QW : DAE' htmlcut[ "td" ]' Q / split data items
( #:' QW ; Q ) / display for editing rows along with how many items
/ saw one line of length 8 was the labels . Stacked ( `ctrl_k ) for later use .
: QW : QW[ & 17 = #:' QW ] / all the real data
+ r / nice to see it flipped too .
: QWE : ncovered[ ( "<" ; ">" ) ]'' QW / throw away all html .
+ QWE[ ; & 0 < +/ #:'' QWE ] / delete all empty rows , flip
: QWE : DRB'' _ssr[ ; " " ; "" ]'' r / delete nonblankspaces .
/ what's left is data
htmldel : { ncovered[ ( "<" ; ">" ) ] x } / need this again so may as well make a fn
htmldel..h : " throw away all html . ( approx ) . "
` $ DAE tokcut[ " " ; ( htmldel ToS ) _dvl "\t\n" ]
/ cleaned up the header line still on the Top of the `.S stack .
/ tapped `f5 to insert them here in `text :
`date `action `quantity `symbol `description `price `amount `comm
/ not all the columns have entries . So :
Schwab[ r ] : QWE
Schwab / f6 to display
/(/ further processing ( date & number conversion , etc ) is outside the topic /
/ FUNCTIONS USED :
VMn MVn tokcut DAE htmlcomdel htmlcut ncovered DRB ToS..d
f4 @ ,// ` $ { ( x ; x , "..h" ) }' VM[ SP ] ToS
VMn />/ { 1 _' ( & x = * x ) _ x : "\n" , x }
VMn..h />/ "split string on \"newline\" character \\n ~ _ci 10 "
MVn />/ { 1 _ ,/ x ,' y }["\n"]
MVn..h />/ "MV using new line chars "
tokcut />/ { ( # x ) _' ( y _ss x ) _ y : x , y }
tokcut..h />/ " split string `y at multi char token `x "
DAE />/ { x @ & 0 < #:' x }
DAE..h />/ " Delete All Empty items of x "
htmlcomdel />/ { ncovered[ ( "<!--" ; "-->" ) ] x }
htmlcomdel..h />/ " delete comments from html string x "
ncovered />/ { y ( ! # y ) _dv/ ,/ to ./: cover[ x ; y ] }
ncovered..h />/ " portion of y not covered within token pair x . cf `htmlcomdel "
cover />/ { QW : ,/ coverraw[ x ; y ]
QW : QW @ & |/ ( 1 >': ; 0 = )@\: 0< nestlevel QW
( _ ( # QW ) % 2 ; 2 ) # QW }
cover..h />/ " indices of top level cover of start/stop tokens `x in string `y "
coverraw />/ { ( + y _ss/: x ) +\: 0 , # *| x }
coverraw..h />/ " indices of occurances of ( starting & ending ) toks in string "
nestlevel />/ { +\ ( ,/ ( ^ x ) # 1 -1 ) @ < ,/ x }
nestlevel..h />/ " level of nestng given list of start and close interval endpoints "
htmlcut />/ { PRTBFs[ ; "</" , x ]' PRTAF[ ; ">" ]' tokcut[ "<" , x ; y ] }
htmlcut..h />/ " splits html y on open and close tags x "
PRTBFs />/ { ( * x _ss y ) # x }
PRTBFs..h />/ "part of x before first occurance of string y"
PRTAF />/ { ( 1 + x ? y ) _ x }
PRTAF..h />/ " Part After First y in x "
DRB />/ { ( x = * y ) _ y : y _di & t & 1 ! t : x = y : x , y }[" "]
DRB..h />/ "Delete Redundant Blanks in string"
ToS..d />/ " *| .S "
ToS..h />/ " Top of .S stack "
/ HTML Table Extraction /\ ==================== /\ ==================== /\
From: Bob Armstrong
Subject: Re: HTML table parsing
Date: Wed, 15 May 2002 18:34:40 -0400
To:
On Wed, 15 May 2002 16:20:28 -0400, David Ness wrote:
> 5/15/2002 3:59:08 PM, Keith Mason wrote:
>
> >Does anyone have K code that parses HTML tables into Kdb tables? I am
> >trying to download financial data from a web site and I need to chop up a
> >table.
>
> I've `parsed' a fair number of different pieces of text coming in in HTML from
> various sources. My experiences would indicate that:
> (1) Sites are highly idiosyncratic;
> (2) The HTML `styles' used are even more idiosyncratic---if that's possible;
> (3) Handling the problem in complete generality is probably:
> (a) quite daunting;
> (b) quite hard; and
> (c) quite unnecessary
> (4) Doing _all_ of the work in K always proved to be a nusciance (for me
> at least---but then I'm not a great K programmer). Things fell much easier
> if I used multi-language environments.
Agreed .
Each time I have to do another , I refine my process I went thru the
last time .
I have several useful functions around to `cut at tokens , delete comments ,
etc , and they are in at least the currently released version of CoSy .
Unfortunately , looking at them right now , they use parsing functions , etc ,
too deeply factored for me to extract and present now .
Maybe within the next couple of weeks as I deal with end of the month stuff
once again .
--
Bob Armstrong -- http://CoSy.com -- 212-285-1864
http://CoSy.com/K/CoSy.htm : New Web structure & Release
http://CoSy.com/Homes : Building Affordable Homes in NYC
2002/05/15 6:03:00 PM
---
The KDB list is managed by majordomo@listbox.com
Send a message there, containing the word 'help', for assistance, e.g. unsubscribing.
|
