Where does R's write function write to?

by Abe Miessler 28. August 2012 06:12

I recently started playing with the programming language R after I discovered Kaggle.  In an effort to help anyone else who is trying to learn this language, I've decided to post resolutions to (very simple) problems that I encounter along my journey.  And our first topic is: Where does R's write function write to?!

I was using the following line of code to do something pretty simple, write a file to disk:

write(results, file="knn_benchmark.csv",ncolumns=1)

But after running that line I was unsure where my file had been written to.  After a quick post to StackOverflow, I found the following method:

getwd()

This method simply returns your current working directory, which is where your file will be written if you don't specify a full path. 

Hope this helps someone!

 

Tags: , ,

StackOverflow.com | r | r-language | kaggle

Making life easier with the DebuggerDisplay attribute in .NET

by Abe Miessler 21. August 2012 06:06

If you have ever worked with a class that holds a collection of items then you have probably seen something like this while debugging:

If you want to see any details about this object you will need to start digging in.  Often a little digging isn't a big deal, but if you are spending a fair amount of time debugging this can quickly become frustrating.  That is why I was so pleased when I came across .NET's DebuggerDisplay attribute.  This handy attribute lets you customize how things will appear when you are debugging.  Say we have the code below which resulted in the screen shot above:

class Order
{
    public List<Item> Items;
}

class Item
{
    public int ItemId;
    public string ItemDescription;
    public decimal Price;
}

class Program
{
    public static void Main()
    {
        Order o = new Order();
        o.Items = GetItems();
        Console.ReadLine();
    }
}

If we take our Order class and modify it like so:

    [DebuggerDisplay("{DebuggerDisplay,nq}")]
    class Order
    {
        public List<Item> Items;
        private string DebuggerDisplay
        {
            get
            {
                return String.Join(", ", Items.Select(t => t.ItemDescription).ToArray());
            }
        }
    }

We begin to get a better picture of the data we are working with:

Ok, now we have an idea of what is in our order without digging into the object.  Lets take it one step further and update our Item class to be a little more informative while debugging:

    [DebuggerDisplay("{DebuggerDisplay,nq}")]
    class Item
    {
        public int ItemId;
        public string ItemDescription;
        public decimal Price;
        private string DebuggerDisplay
        {
            get
            {
                return String.Format("{0} - {1:C}",ItemDescription,Price);
            }
        }
    }

Now when we are debugging we will see the following:

 

Clearly there is a lot more information available without digging into the various objects. 

 

In summary, you can use the DebuggerDisplay attribute in a number of ways to make your life easier while debugging.  You also have the options of using the DebuggerDisplay attribute without adding a DebuggerDisplay member to your class as I have in my examples.  I like doing this because it allows you more control over what is displayed and just seems cleaner compared to the alternative.  If you are tired of clicking through objects while debugging I highly recommend you give the DebuggerDisplay attribute a try!

Tags: , ,

Introducing SharePoint to the StackExchange family!

by Abe Miessler 21. April 2011 05:58

I have been a slave to StackOverflow for some time now.  In fact I recently cracked the 10,000 reputation mark that grants me access to the moderator tools (yay me!).  Feel free to take a look at my account:

profile for Abe Miessler at Stack Overflow, Q&A for professional and enthusiast programmers

Anywho... at my new job I have been thrust into SharePoint development with a vengence.  As the only .NET/SharePoint programmer there and having no prior experience, the learning curve has been a bit steep...  Luckily for me, the StackExchange family has recently added a SharePoint site!  You can find it at SharePoint.StackExchange.com.

It has a brilliant community of IT professionals that have been able to quickly answer any SharePoint related questions I've had.  If you work with SharePoint and you have questions regarding usage, administration or development in SharePoint bring them to SharePoint.StackExchange.com!

Tags:

SharePoint

Adding javascript files to your SharePoint 2010 web part.

by Abe Miessler 12. April 2011 07:31

For something so simple, this problem took me quite a while so I wanted to share how I resolved it.  In my situation I wanted to add jQuery and a few jQuery plugin files to be part of my visual web part.  This web part was going to be part of a wsp that would be shipped out to SharePoint environments that I had no control over, so simply including them in the master page was not an option.

 

  1. Right click on your project file and select Add->SharePoint "Layouts" Mapped Folder
  2. In side of the Layouts Mapped folder create the following file structure: YourProjectName/js
  3. Add your js files to the folder you just created in your project
  4. Inside of your ascx file for you visual web part create the ScriptLink tag

             <SharePoint:ScriptLink ID="ScriptLink1" runat="server" Name="YourProjectName/js/jquery.min.js" />

If you have jQuery plugins, place their ScriptLink  tags AFTER the jQuery tag

 

You should now be in business.  If you are havinig problems check the following directory to make sure your scripts are being deployed:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\LAYOUTS\YourProjectName\js

 

 

Tags: ,

ASP.NET | jQuery | SharePoint

RegEx DOS attack - Regular Expressions: Now you have 3 problems

by Abe Miessler 26. October 2010 03:19

I recently came across an article in MSDN magazine called Regular Expression Denial of Service Attacks and Defenses.  This article detailed a relatively new breed of Denial of Service (DOS) attack that exploits a feature of Nondeterministic Finite Automaton regex engines.  This is the type of regex engine used by .NET, Perl, Pyton, Ruby and PHP to name a few.  One thing that is so disturbing about this type of attack is how easy it is for malicious web users to find a vulnerability in your site if you are doing any kind of client side validation.  I for one have lived by the motto, "Use client side validation for user experience and server side validation for security."  Unfortunately with this type of attack sending your regex to the client will make it much easier for malicious users to find vulnerabilities in your site.

The problem regexs are ones that have something called exponential complexity.  This basically means that the number of paths that must be evaluated increases exponentially with every character added.  You run into this situation when you create a regex that has a grouping expression with repetition that is repeated itself.  Typically this basic example is given: ^(\d+)+$. 

For the purposes of our example we will use a slightly more complex regex that I found on RegexLib.  This regex is used to validate email addresses (nested repetition has been bolded): ^([0-9a-zA-Z]([-.\w]*[0-9a-zA-Z])*@([0-9a-zA-Z][-\w]*[0-9a-zA-Z]\.)+[a-zA-Z]{2,9})$

I suspect that this is a widespread problem and I plan on reviewing my own code to see if I have put any vulnerabilities out there.  To demonstrate this vulnerability I have put together a small application to run some tests against.  First I created an aspx page with a TextBox, a RegularExpressionValidator, a Label and a Button:

 

 

For the purpose of this demonstration I have set "CausesValidation" to false in my Button.  This control is only setup like this so that we can time processing time on the server side.  Below is the code for btn_SubmitClick:

 

 

 

In this code we are just measuring how long it takes for the validation to take place.  Lets give our page a run:

 

I know... not that exciting.  But now lets take a look at the rendered HTML

 

 

Notice the area highlighted in red.  This serves to make the life of any evil doers much easier.  Since the regex is on the clientside they can examine each one and look for weaknesses.  Think they will face the same delay on the client side? Not if they use something like BurpSuite to yank the javascript.  Think you can limit the number of characters they enter into the text box?  Again not if they use something like BurpSuite...  Now that we have established that you are screwed lets take a look at what some sample inputs do. 

First lets try 15 characters:

.012 seconds? Piece of cake...  Lets try 20:

.39 seconds?  Still not that scary... Lets try 25:

12.542 seconds! Now it's getting a little scary.... Lets push it to 28:

93.381 seconds...  And lets take a look at my CPU during that time:

 

90 seconds of hell.  I'll leave it to you to go beyond 28 characters, but I waited for quite a while at 30 characters before I gave up. 

As you can imagine a few malicious users or a small bot net could bring a website to it's knees if they have vulnerable regexs out there.  Unfortunately I don't know of a quick fix for this problem.  The best solution I've seen so far involves finding all vulnerable Regular Expressions and rewriting them.  The article mentioned at the beginning of this post goes over a good "Regex Fuzzer" (a way to find problem regular expressions), and it's definitely worth looking at.

Good luck!

Tags:

web application security | denial of service | DOS | Regular Expressions

Validating currency input in ASP.NET

by Abe Miessler 13. September 2010 16:02

While doing some input validation for a web application I am working on I attempted to use ASP.NET's CompareValidator to do some pretty basic validation for a currency input.  I was surprised to find that it kept saying my input wasn't valid any time I included the "$" symbol.  Below is an example of how I implemented the control:

<asp:CompareValidator ID="vld_Cash" runat="server"
     ControlToValidate="CashTextBox"
     Type="Currency"
     Operator="DataTypeCheck"
     ValidationGroup="vld_Update"
     ErrorMessage="You must enter a currency for 'Cash'. Example: $500" />

Thinking that this might be some bizzare intended behavior from Microsoft I decided to take a look at the documentation.  According to Microsoft the Currency data type is:

    "A decimal data type that can contain currency symbols."

I did some more poking around and it looks like i'm not the first person to have this problem.  From what I've read the best work around for this is to create a RegularExpressionValidator that will allow input that is formatted as currency.  When creating the regular expression you will be using with your RegularExpressionValidator remember to use JScript compliant regex.  I made the mistake of using .NET regex and when it tried to validate on the client side it threw an exception.  If you create a JScript regex it should work fine on both client and server side.  Below is the RegularExpressionValidator that I ended up using to accomplish what ASP.NET's CompareValidator couldn't:

<asp:RegularExpressionValidator ID="rev_CashTextBox" runat="server"          
        ValidationExpression="^\s*\$?\s*(?!\d{4,},)(\d|\d{1,3},(?=\d{3})(?!\d{4}))*(\.\d{1,2})?\s*$"
        ControlToValidate="tb_Cash"
        ValidationGroup="vld_Edit"
        ErrorMessage="You must enter a currency value for Cash'. Examples: 100.00, $100" />

This allows for a number of different American currency formats to be submitted.

Here is a link to the bug I submitted to Microsoft, hopfully they will find a resolution soon.

Tags:

ASP.NET

Deleting duplicate rows in SQL Server

by Abe Miessler 27. July 2010 18:03

I came across this very slick method for deleting duplicate records while poking around on StackOverflow today.  I've used a variety of methods in the past for deleting dupes, but nothing quite as clean as this.

The SQL below would work for a table in the following format, you'll need to adjust as necessary.  Please note that this will only work if id is a Primary key.

 

MyTable: id, Col1, Col2, Col3

 

And the SQL:

 

DELETE MyTable 
FROM
MyTable
LEFT OUTER JOIN
(
   SELECT MIN
(id) as id, Col1, Col2, Col3
   FROM
MyTable
   GROUP BY
Col1, Col2, Col3
) as KeepRows ON
   
MyTable.id= KeepRows.id
WHERE
   
KeepRows.id IS NULL

Hope this helps someone!

Tags:

SQL | StackOverflow.com

Implementing a sequential pulsate effect using jQuery.

by Abe Miessler 22. June 2010 00:58

I was recently working on a webpage where I wanted to have a collection of tabs at the top of the page pulsate one after the other.  Had to do a bit of digging around (and get some help from a friend of mine who happens to be a jQuery expert) so I thought I would post it up here incase anyone else runs into this.  Once it was done it was remarkably little code, yet another example of why I'm falling in love with jQuery and jQuery UI.

<html>
    <head>
    <link href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/themes/base/jquery-ui.css"
        rel="stylesheet" type="text/css" />

    <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4/jquery.min.js"></script>

    <script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/jquery-ui.min.js"></script>
        <script>
              $(document).ready(function () {
                  $("#myTabs").tabs({ fx: { opacity: "toggle"} });
                  recurivePulsate($("#items").find("li")[0]);
              });

              function recurivePulsate(elem) {
                    $(elem).effect("pulsate", { times: 1 }, 500, function () {
                        recurivePulsate($(elem).next());
                    });
                }
        </script>
    </head>
    <body>
        <div id="myTabs">
            <ul id="items">
               <li><a href="#first">First</a></li>
               <li><a href="#second">Second</a></li>
               <li><a href="#third">Third</a></li>
               <li><a href="#fourth">Fourth</a></li>
            </ul>
            <div id="first">Data for First!</div>
            <div id="second">Data for Second!</div>
            <div id="third">Data for Third!</div>
            <div id="fourth">Data for Fourth!</div>
        </div>
    </body>
</html>

Tags:

jQuery

Proper input length validation.

by Abe Miessler 3. March 2010 09:07

When creating a web application that accepts input from users one important step to ensure that you application is secure, is assigning a maximum length to user input.  Keep in mind that input length validation is just the tip of a very large iceberg when it comes to web application security, but it is a good first step.  It's safe to say that if someone is trying to put in 1200 characters for their first name that something fishy is going on.

As much as I love ASP.NET I've found that it can easily lull some programmers into a false sense of security when it comes to input length validation.  To demonstrate what I am talking about I have put together a small application that will show how simple it is to pass longer than expected input to an application that would appear (to the uninformed programmer at least) to limit input length.

To begin I have created the basic aspx page below.  Notice the MaxLength attribute that is highlighted in red:

This is where our security troubles often begin....  It seems like this MaxLength is telling us that it won't let anything over 9 characters get passed to our beloved server.  Don't believe it!  IT'S LYING!!!  To understand why the MaxLength attribute dosn't do everything it claims it does lets go ahead and run out application and take a look at the rendered HTML:

As you can see, our seemingly invincible asp:TextBox control has been rendered as a standard html input with a maxlength of 9.  This is bad because this on the client's side, so the client can change whatever they would like and send it back to the server.  To show you what i'm talking about I am going to make use of a neat application called Burp Suite.  Burp Suite is a collection of tools that can be used for attacking web applications.  If you are at all interested in web security I recommend you download it and check out some of it's features.  For the porpouse of this post we will be using Burp Proxy, which is a HTTP Proxy server that allows you to intercept, inspect and modify the raw http traffic being passed in both directions. 

First I will go to the webpage with the vulnerability without intercepting anything.  I attempt to add more than 9 characters and, as advertised, I am not able to. 

Now I will begin capturing http traffic and submit the search text to the server.  This is where things get interesting...  Notice the red box in the image below:

So at this point burp suite has captured this HTTP request, showed me what params are in it (as well as letting me look at and mess with a whole lot of other info in the request).  It is basically in a holding state at this point until I have modified what I want and decide to forward it to the server.  So next I will change the value of tb_SearchBox and forward the request.  See below:

As you can see i've added what ever I wanted to the value of tb_SearchBox and this will now be passed to the server when I click the "forward" button.

Alright... so now that we have an idea of why this is a vulnerability how do we fix it?  The answer is by adding server side input validation.  I've found that the simplest way to do this is to compare the text length to the MaxLength of the TextBox control.  If the former is longer than the latter then your website is under attack.  Period.  What you do at that point is up to you but I recommend at a minimum doing what the comments in the IF statement from the image below suggest.

 

I hope this helps to make your website just a little more secure.  Remember, always be wary of built in ASP.NET validation and when in doubt, go ahead and check it on the server side.

 

 

 

 

 

Tags: ,

web application security

Adding reCAPTCHA to your BlogEngine.net site.

by Abe Miessler 3. March 2010 07:15

After being hit by a barage of spam comments I decided it was time to look at adding CAPTCHA to AbeMiester.com

NOTE TO GOOGLE: I am in no way affiliated with any organizations trying to buy gold, sell miniture bicycles(wtf?) or hawk a natural cure for acne.

I did some research and finally settled on reCAPTCHA for my anti-Bot needs.  reCAPTCHA is a free CAPTCHA service that actually helps to digitize aging books, newspapers and radio broadcasts for future generations.  The way they do this is actually quite facinting and I recommend you read more about it.

While the process of adding reCAPTCHA to your average website is quite simple, I found that adding it to BlogEngine.NET was fairly difficult.  Luckily for me Filip Stanek did an excellent post on integrating reCAPTCHA into your BlogEngine.net website.  He provides instructions on how to do it quick and dirty as well as details as to exactly what was changed.

If you've been having spam comment problems on your BlogEngine.net site I'd highly recommend you looking into adding reCAPTCHA!

Tags: , ,

spam | bots | CAPTCHA

Powered by BlogEngine.NET 1.6.0.0