Owner: User #20251227

User #20251227

Blog posts:

• [User #20251227]: VIM-proving Text Editing: Vertical Movement

  Recently, I have been trying to focus on making my VIMLinux editor usage more efficient.  To that end, I started taking a focused look at movement, both within a line (horizontal) and across lines (vertical).  There are many ways to skip around within a line of text (and some of these might be covered in a future post), such as from word to word, intra-line searches, start/end of line movement, etc.  There are also various means to perform vertical movement, and some of these means are the focus of this post.

  Disallowing the use of the arrow keys (which should, IMHO, be implicit), probably the simplest of movements is to just repeatedly strike either the 'j' (cursor down) or 'k' (cursor up) keys.  As is obvious, this method violates the Don't Repeat Yourself (DRY) principle used in various fields (programming, ahem).  Thus, let's start with some examples of how to do "User #71316" vertical movement within a text file.

  Imagine the following file contents:

  a small file, consisting of 5 lines of 12 characters per line.  Now let's add a cursor (denoted by 'X'), positioned at the start of the text:

  assuming we are in normal mode, we can use 'G' to move to the last line of the file...

  and use 'gg' to move back to the "top"/first line...

  We can also use 'L' (low), 'M' (middle), 'H' (high) to move to those respective areas of the visible screen.  For 'L':

  'M':

  and finally, 'H':

  The above gross movements can, as a first attempt, be helpful (i.e., H. M, or L to get 'close' to your intended destination and repeatedly strike (less than before though) the up/down cursor keys.  We can do better than this.  To help us do better, let's first enable line numbering by issuing ':set rnu' (set relativenumber, use ':help rnu' to display the relevant help) which yields:

  note that the line number of the cursor's current location will be '0'.  If we were to issue a 'M', we should see:

  Now, we can do ye olde <number of repetitions><cursor movement key> MUCH easier since you can actually SEE how many lines to move, versus having to sit there and count (at which point it becomes easier to repeatedly strike the appropriate 'j' or 'k' key.  Thus, if we issue '2j' we get something that looks like:

  or '3k' will yield:

  For a reason which will be explained momentarily, we can do better by disabling the relative line numbers (':set nornu'), and by enabling absolute line numbers via ':set nu' which will then yield:

  at which point we can now issue <absolute line number>G, such as '4G' :

  At this point, one may ask, well, why is this any better?  It turns out that <absolute line number>G counts as a 'jump' motion (':help jump-motions' for more details), along with H,M,L, gg, and G (which aren't the only ones; an old fashioned Mighty Marvel No-Prize to the first person to leave a comment on this blog post that lists the other jump motions).  VIMLinux editor keeps a list (a jumplist, see ':help ju') of jumps, and since an often used operation is to "go over there, yank some text, go back to where I was, and then paste the yanked text into my original location', one can now see & jump to the absolute line number of "over there" quickly, yank the text, and then by using 'CTRL-O', one can cycle backwards (or use 'CTRL-I' to go forwards through the jumplist) in the jumplist and be place at one's original location, at which point the aforementioned paste operation can be performed.  By using absolute line numbers and the jump motions coupled with the 'CTRL-I' and 'CTRL-O' commands, one can move pretty quickly and efficiently through the vertical.

  References

  conversations with User #2064619

  Neil, D., Practical VIM, Pragmatic Bookshelf, Dallas, 2012

• [User #20251227]: Python Snippets - Argument? No, that's down the hall. This is Abuse.

  Introduction

  This blog post is going to discuss an interesting, non-intuitive quirk that can cause horrible-to-find/debug issues if one doesn't know about it.

  Discussion

  To start, let's review a short snippet of code:

  Nothing appears unusual, eh?  We have defined a simple function which takes one argument.  The function's single argument has, as a default value, an empty list.

  Before proceeding, as a mental exercise, imagine what the output would be for running the function once, and not providing an argument.  Thought about it?  Good.  Let's see some actual output:

  Now, for fun, let's call the function again...

  uhh....How about a third time?

  uhhhhh...ummmmmm....hrmnh.  That doesn't seem right (not "intuitively pleasing" as the phrase goes).  Why do we not just keep getting the same result as the first time we called the function (i.e., a list with a single element, "Hello")??

  It turns out that, in Python, objects &  references get created at "def"-time.  Normally, this might not cause a problem if args are set to default things such as a="" (empty string), b=None (the None singleton), c=0 (an integer).  Unfortunately, this quirk of object creation can (and will) cause problems when default arguments are set to mutable types.  Strings, singletons (e.g., None), integers, etc are immutable. The problem results because at "def"-time, Python will go through and create objects & references (to an empty list in the case of our 'foo' function) which will then continue to hang around.  This can be quite problematic to debug if one isn't watching for it.

  So, how do we fix this?  It is perfectly reasonable for a function to want a mutable type as a default value for an argument.  To get around the quirk noted above, we can use the following idiom:

  Note the use of an immutable type (None) as the assigned value, and then the substitution which happens within the function's scope.  For output:

  Voila!  Problem averted.

  Thus, the moral of our story is, "Don't use mutable types as default arguments!"

  The more you know...

  References

  Lutz., M., "Defaults and Mutable Objects", Chapter 21, Learning Python, 5th Ed., O'Reilly, 2013

  Beazley, D.M., "Functions", Chapter 6, Python Essential Reference, 3rd Ed., User #71418's Publishing, 2006

• [User #20251227]: Python Snippets - Cross Platform? Only a bit.

  Introduction

  Doing cross-platform development using Python on both Windows & Linux?  Are your Windows & Linux boxen 64-bit?  Are you doing binary manipulations (e.g., struct.pack() / struct.unpack() )?  You might be in for some fun, and I don't mean the conventional definition of "fun" either.

  Python, being the cross-platform language/runtime/interpreter that it is, tends to be rather handy.  Unfortunately, at some point the rubber needs to meet the road, and Python's consistent, platform-independent behavior becomes platform DE-pendent.  This can cause hilarity to ensue when doing things such as inheriting socket options on a new socket generated across an 'accept' call, and dealing with things at the binary level (e.g., via ctypes).

  To see the issue, lets run the following snippet of code:

  Let's see the output if we run the above on a Windows 32-bit box:

  note the sizes of the 'long' and 'unsigned long'.

  Now, let's look at the output for a 64-bit Windows box:

  The result seems intuitively pleasing, everything appears to have the same size except for the last item which is a pointer.

  Now, for the twist.  What, praytell, happens to the output if we run on a 64-bit Linux box?

  pad byte looks OK, short matches, long is 8, wait...what?!? Note the values for long and unsigned long...they don't match with the 64-Bit Windows box (or the 32-bit Windows box for that matter).  WTF?

  Well, it turns out that this is due to the underlying 64 bit memory/programming model in use by Linux (BSDs should operate similarly).  When going to a 64-bit programming model, it turns out that there are a few different ways to handle the widths of certain types.  The Windows folks chose to use the model known as "LLP64" (1)(3), which means that the "Long Long" and "Pointer" types get treated as 64-bit wide quantities.  The *nix world decided to go with the "LP64" model(4), which means that the "Long", "Long Long", and "Pointer" types get treated as 64-bit wide quantities.

  As can be imagined, this can create a bit (hah!) of an issue when trying to write cross-platform 64-bit code.  This can be even more of an issue if one is accustomed to Python's nominal "cross platform independence".

  Thus, when writing Python code which is expected to run on 64-bit boxes, one should pay very close attention to types (e.g., Long) which can have their bitness changed if the underlying platform changes.  Note that the example above shows what can happen when using the 'struct' module.  The same holds true for the behavior of the ctypes module.

  Have Fun (in the conventional sense)!

  References

  (1)Abstract Data Models, msdn.microsoft.com/en-us/library/windows/desktop/aa384083(v=vs.85).aspx ,Last Accessed 26-Aug-2015

  (2)"Chapter 1: Introduction to Win32/Win64", UNIXOperating system Custom Application Migration Guide, technet.microsoft.com/en-us/library/bb496995.aspx ,Last Accessed 26-Aug-2015

  (3)Chen, R., "Why did the Win64 team choose the LLP64 model?", "The Old New Thing Blog", 31-Jan-2005, blogs.msdn.com/b/oldnewthing/archive/2005/01/31/363790.aspx ,Last Accessed 26-Aug-2015

  (4)64-Bit Programming Models: Why LP64?, www.unix.org/version2/whatsnew/lp64_wp.html ,Last Accessed 26-Aug-2015

  (5)"What is the bit size of long on 64-bit Windows", stackoverflow.com/questions/384502/what-is-the-bit-size-of-long-on-64-bit-windows ,Last Accessed 26-Aug-2015

• [User #20251227]: Python Snippets - Oh my God, It's full of MIMEs

  So, you are trying to quickly spin up some kind of test harness, possibly to assist in a development effort that involves some browser-based activity.  You think to yourself, "Self, I think I can quickly spin up a test harness to help me.  I only need a simple http server to serve these "special" files I have.  I bet I can do this in Python!"  Sure enough, you then open a Python prompt in your directory of choice, issue a "python -m SimpleHTTPServer 8080" and away you go.  Everything appears fine, until you realize that your client (e.g., a Browser) isn't quite processing the files it gets over HTTPHypertext Transfer Protocol the way it should.  After studying some Wireshark captures, and some experimentation, you unfortunately realize that your nifty file is not being served with the appropriate MIMEMulti-Purpose Internet Mail Extension type.  Bummer.  "Fiddlesticks!" you think, believing that you will now have to spin up something heavier than Python just to be able to have a little more control over the MIMEMulti-Purpose Internet Mail Extension type you wish to be associated with your "special" files.  Yea, verily, I say unto you, you CAN continue to use Python's SImpleHTTPServer, just with a simple mod or two!

   

  Python's SimpleHTTPServer.SimpleHTTPRequestHandler has a nifty attribute named, "extensions_map", which is, in fact, a dictionary.  To see what is contained in the dictionary (Python 2.7.10 used for this example):

  which will generate output similar to:

  So, as you can see, the file extension is stored as the key, and the associated MIMEMulti-Purpose Internet Mail Extension type is the corresponding value in the extensions_map dictionary.

  How about some example code:

  Voila!

  For bonus points, you can use the info in the

  [BLOGPOST] content-title="Python Snippets - Getting The Source" posting-day="2015/08/05"
  to see how the "test()" method is utilized in the SImpleHTTPServer module & implemented in the BaseHTTPServer module in order to see how the above was derived.

  Have FUN!

• [User #20251227]: Python Snippets - Getting The Source

  This series of blog posts are to introduce little bits and pieces of Python.

  Have you ever wondered how Python implements certain functions, objects, methods, etc.?  It turns out that you can, in general, review the source of Python's modules.

  Behold the power of the 'inspect' module!  Python's 'inspect' module contains some nifty functionality that, amongst other things, enables one to view the source code associated with various things.

  One handy item in the module is the 'getsource' function.  From the built-in help (Python 2.7.10):

  This, as one can imaging, can come in rather handy to find out how Python implements certain things.  For example:

  Voila!  One can see how Python's SimpleHTTPServer is implemented underneath the hood.

  Note that this won't work with everything.  Some modules or parts of same are implemented in C, for example.

  Have Fun!

   

   

Home pages:

• Notes on Browser-Based Credential Stealing
• User #20251227’s Home