Announcement

Collapse
No announcement yet.

A response to YOhio

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • A response to YOhio

    YOhio asked me why I haven't done any geek statistical posts in honor of the upcoming season. Lack of inspiration I guess. However, I am thinking I should help make geeky statistical analysis easier for the masses.

    So here you go. My computer model program. Right now it is a raw margin of victory model but it is easy to change. I hope you find it useful YOhio. All you need is this program and the list of 1A and 1AA teams and the scores. It is not the fastest thing ever since it just using perl and not very efficient (in multiple ways) but it is only like a 40 seconds run time on a good computer.

    Teams

    Scores 2008

    Code:
    #!/usr/bin/perl 
    use Statistics::Regression;
    
    open (FILE,"team.txt") or die "Can't open; $!\n";
    $_ = <FILE>;
    
    my %t;
    my %team;
    my $i = 0;
    while (<FILE>) {
      my ($tname) = split("\n",$_);
      $tname =~ s/\s+$//;
      $team{"$i"} = $tname;
      $t{$tname} = $i++;
    }
    close (FILE);
    
    my $skip = "244";
    
    my $yvar = "mov";
    my @xvar = ("hf");
    foreach $key (sort keys %t) {
      if ($t{$key} != $skip) {
        push(@xvar,$t{$key});
      }
    }
    
    my $reg = Statistics::Regression->new( "Computer Model",\@xvar);
    
    open (FILE,"score_08.txt") or die "Can't open; $!\n";
    $_ = <FILE>;
    
    my %wins;
    my %loses;
    
    while (<FILE>) {
      my $net = substr($_,72,1);
      $net =~ s/\s+$//;
      my $visitor = substr($_,10,27);
      $visitor =~ s/\s+$//;
      my $home = substr($_,41,27);
      $home =~ s/\s+$//;
    
    
      my $scored = substr($_,37,3);
      my $allowed = substr($_,68,3);
      my $mov = abs($scored - $allowed);
      
      my $win = "h";
      if ($scored > $allowed) {
        $win = "v";
      }
      
      my $hf = 1;
      if ($net ne "" && $net ne " ") {
        $hf = 0;
      }
      elsif ($win eq "v") {
        $hf = -1;
      }
    
      # Adjustments to neutral fields
      if (($home eq "Kent St" && $visitor eq "Boston College")
        || ($home eq "Washington St" && $visitor eq "Oklahoma St")
        || ($home eq "Colorado" && $visitor eq "Florida St")) {
        my $hf = 1;
        if ($win eq "v") {
          $hf = -1;
        }
      }
    
      if (($home eq "Arkansas" && $visitor eq "Louisiana-Monroe")) {
        my $hf = -1;
        if ($win eq "v") {
          $hf = 1;
        }
      }
    
      # End adjustments to neutral fields
      
      
      if ($t{$visitor} ne "" && $t{$home} ne "") { 
          
        if ($win eq "h") {
          my $xvar = $t{$home};
          $wins{$xvar} += 1;
          
          $xvar = $t{$visitor};
          $loses{$xvar} += 1;
        }
        else {
          my $xvar = $t{$home};
          $loses{$xvar} += 1;
    	
          $xvar = $t{$visitor};
          $wins{$xvar} += 1;
        }
    
        my @x = ($hf);
        foreach $key (sort keys %t) {
          if ($t{$key} != $skip) {
    	if ($key eq $home) {
    	  if ($win eq "h") {
    	    push(@x,1);
    	  }
    	  else {
    	    push(@x,-1);
    	  }
    	}
    	elsif ($key eq $visitor) {
    	  if ($win eq "v") {
    	    push(@x,1);
    	  }
    	  else {
    	    push(@x,-1);
    	  }
    	}
    	else {
    	  push(@x,-0);
    	}
          }
        }
        $reg->include($mov,\@x);
      }
    }
    
    my @theta = $reg->theta();
    
    sort @tmp = sort {$a <=> $b} @theta;
    my $rmin = $tmp[0];
    
    my %r;
    $r{$skip} = 0.0;
    for (my $i = 1; $i < $#xvar; ++$i) {
      $r{$xvar[$i]} = $theta[$i] - $rmin;
    }
    
    
    
    print "NCAA Football: 2008\n";
    print "Computer Rating Model: Margin of Victory (RAW)\n";
    print "Only Division 1A and 1AA games used\n\n";
    printf("%-5s %-22s %7s %3s %3s\n","Rank","Team","Rating","W","L");
    my $i = 1;
    foreach $key (sort {$r{$b} <=> $r{$a}} keys %r) {
      printf("%-5d %-22s %7.3f %3d %3d\n",$i++,$team{$key},$r{$key},$wins{$key},
        $loses{$key});  
    }

  • #2
    Why the 4 neutral site game adjustments?

    Why didn't you include games like the OU/Texas and all the others?

    And why is Arkansas v. Louisiana-Monroe reversed?

    Comment


    • #3
      Originally posted by beefytee View Post
      Why the 4 neutral site game adjustments?

      Why didn't you include games like the OU/Texas and all the others?

      And why is Arkansas v. Louisiana-Monroe reversed?
      I don't remember ... If I had to guess those are the pseudo neutral field games. Like Boston College played Kent St in Cleveland, Ohio. I would adjust it so the home team was Kent St. instead of it being neutral. OU/Texas I imagine I left as a neutral field game.

      Comment


      • #4
        Originally posted by pelagius View Post
        However, I am thinking I should help make geeky statistical analysis easier for the masses.
        Thanks man! You're like the Johannes Gutenberg of Perl scripts. For so long we, the ignorant masses, have been unable to do geeky statistical analysis. But you've unlocked the door for us. For this I'm certain you'll face scorn amongst your peers. Notice that Lebowski has kept silent.

        Comment


        • #5
          Originally posted by YOhio View Post
          Thanks man! You're like the Johannes Gutenberg of Perl scripts. For so long we, the ignorant masses, have been unable to do geeky statistical analysis. But you've unlocked the door for us. For this I'm certain you'll face scorn amongst your peers. Notice that Lebowski has kept silent.
          Is it early morning in China? These threads aren't the same with you sleeping during our daily posting rush.

          Comment


          • #6
            Originally posted by Fiyero View Post
            Is it early morning in China? These threads aren't the same with you sleeping during our daily posting rush.
            It's not early, early. Beijing is twelve hours ahead of Eastern Time.

            Comment


            • #7
              Pelagius, I'll see your geek and raise you two nerds. Here's an excerpt of my Excel VBA program to go out to the web, download play by play data and parse it into various fields. The messageboard software and propriety concerns (wink wink) prevent me from pasting the program in its entirety.

              Code:
              Sub PlayParse()
              '
              '
              Season = Year(Sheets("WeekMenu").Cells(22, 3))
              Week = Sheets("WeekMenu").Cells(22, 1)
              GameDate = Sheets("GameMenu").Cells(22, 3)
              
              Cells(1, 1).Select
              Do While FindString(Selection.Value, "Drive #", 1) = 999
                  Selection.Offset(1, 0).Select
              Loop
              
              Selection.Offset(2, 0).Select
              
              Range(Selection, Selection.End(xlDown)).Select
              PlayRowStart = Selection.Row
              EndPlays = Selection.Rows.Count + Selection.Row - 1
              
              
              For x = PlayRowStart To EndPlays
                  
                  Cells(x, 1).Select
                  
                  If NoPlayFind(Selection) Then GoTo NoPlay
                          
                  Sheets("Parsed Data").Cells(OutputRow, 1) = Season
                  Sheets("Parsed Data").Cells(OutputRow, 2) = Week
                  Sheets("Parsed Data").Cells(OutputRow, 3) = GameDate
                  Sheets("Parsed Data").Cells(OutputRow, 4) = GameID
                  Sheets("Parsed Data").Cells(OutputRow, 5) = PlayID(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 6) = Quarter(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 7) = TeamWithBall(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 8) = DriveStartQtr(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 9) = DriveStartClock(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 10) = DriveNum(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 11) = PlayNum(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 12) = BallPos(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 13) = Down(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 14) = YdsToGo(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 15) = RedZone(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 16) = HomeScore(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 17) = AwayScore(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 18) = PlaysInDrive(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 19) = YdsInDrive(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 20) = DriveTOP(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 21) = DriveResult(Selection)
                  
                  Sheets("Parsed Data").Cells(OutputRow, 22) = Run(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 23) = Pass(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 24) = FGAtt(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 25) = FGMade(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 26) = Touchdown(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 27) = PATAtt(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 28) = PATMade(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 29) = TwoPtAtt(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 30) = TwoPtMade(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 31) = Kickoff(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 32) = Punt(Selection)
                  
                  Sheets("Parsed Data").Cells(OutputRow, 33) = Gain(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 34) = Completion(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 35) = Sack(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 36) = Interception(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 37) = Fumble(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 38) = Penalty(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 39) = PenaltyYds(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 40) = Declined(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 41) = PlayNegated(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 42) = PenaltyOn(Selection)
                  
                  Sheets("Parsed Data").Cells(OutputRow, 43) = FirstDownMade(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 44) = ThirdDownAtt(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 45) = ThirdDownMade(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 46) = FourthDownAtt(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 47) = FourthDownMade(Selection)
                  
                  Sheets("Parsed Data").Cells(OutputRow, 48) = PuntYds(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 49) = KickoffYds(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 50) = Touchback(Selection)
                  
                  'Sheets("Parsed Data").Cells(OutputRow, 51) = PuntInside20(Selection)
                  'Sheets("Parsed Data").Cells(OutputRow, 52) = PuntInside10(Selection)
                  'Sheets("Parsed Data").Cells(OutputRow, 53) = PenaltyOn(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 54) = Safety(Selection)
                  Sheets("Parsed Data").Cells(OutputRow, 60) = PlayText(Selection)
                  If (HomeAbbrevFlag = True) And (AwayAbbrevFlag = True) Then
                      BYU = ""
                  Else: Call FindAbbrev(Selection)
                  End If
                  OutputRow = OutputRow + 1
              NoPlay: Next x
              
              End Sub
              Function NoPlayFind(ByRef Selection As Range) As Boolean
                  
                  NoPlayFind = False
                  
                  If FindString(Selection, "drive start", 1) <> 999 Then NoPlayFind = True
                  If FindString(Selection, "Timeout", 1) <> 999 Then NoPlayFind = True
                  If Application.WorksheetFunction.Min(FindString(Selection.Value, "rush for ", 1), _
                                                       FindString(Selection.Value, "rush over ", 1), _
                                                       FindString(Selection.Value, "rush to ", 1), _
                                                       FindString(Selection.Value, "rush quarterback ", 1), _
                                                       FindString(Selection.Value, "rush option ", 1), _
                                                       FindString(Selection.Value, "rush up ", 1), _
                                                       FindString(Selection.Value, "sacked ", 1), _
                                                       FindString(Selection.Value, "pass ", 1), _
                                                       FindString(Selection.Value, "pass to ", 1), _
                                                       FindString(Selection.Value, "pass for ", 1), _
                                                       FindString(Selection.Value, "field goal attempt ", 1), _
                                                       FindString(Selection.Value, "kick attempt ", 1), _
                                                       FindString(Selection.Value, "kickoff ", 1), _
                                                       FindString(Selection.Value, "punt ", 1), _
                                                       FindString(Selection.Value, "PENALTY", 1)) = 999 Then NoPlayFind = True
                  If Selection.Hyperlinks.Count = 1 Then NoPlayFind = False
              
              End Function
              Function FindString(ByRef TextLine As Variant, KeyString As String, StartPos As Integer) As Integer
              
              If Selection.Hyperlinks.Count = 0 Then
                  If Selection.Count = 1 Then
                      TextString = TextLine
                  Else: TextString = TextLine(1, 1)
                  End If
              End If
              
              If Selection.Hyperlinks.Count = 1 Then
                  If Selection.Count = 1 Then
                      If Selection.Value <> TextLine Then
                          TextString = TextLine(1, 1)
                      End If
                  Else: TextString = TextLine(1, 1)
                  End If
              End If
              
              'If Selection.Hyperlinks.Count = 1 Then
              '    If Selection.Value <> TextLine(1, 1) Then
              '        TextString = TextLine(1, 1)
              '    End If
              'End If
              
              FindString = StartPos
              KeyLen = Len(KeyString)
              TextLen = Len(TextString)
              
              While Mid(TextString, FindString, KeyLen) <> KeyString
                  FindString = FindString + 1
                  If (FindString + KeyLen) > (TextLen + 1) Then
                      FindString = 999
                      Exit Function
                  End If
              Wend
              
              End Function
              Function FindNumber(ByRef TextLine As Variant, KeyNumber As Integer, StartPos As Integer) As Integer
              
              If Selection.Hyperlinks.Count = 0 Then
                  If Selection.Count = 1 Then
                      TextString = TextLine
                  Else: TextString = TextLine(1, 1)
                  End If
              End If
              
              FindNumber = Application.WorksheetFunction.Max(13, StartPos)
              KeyLen = Len(Application.WorksheetFunction.Text(KeyNumber, "0"))
              TextLen = Len(TextString)
              
              WrongNumber:
              
              While Mid(TextString, FindNumber, KeyLen) <> Application.WorksheetFunction.Text(KeyNumber, "0")
                  FindNumber = FindNumber + 1
                  If (FindNumber + KeyLen) > (TextLen + 1) Then
                      FindNumber = 999
                      Exit Function
                  End If
              Wend
              
              If (Mid(TextString, FindNumber - 1, 1) = " ") And (FindNumber < TextLen) Then
                  FindNumber = FindNumber + 1
                  GoTo WrongNumber
              End If
              
              End Function
              Function DriveStartClock(ByRef Selection As Range)
              
              If Selection.Hyperlinks.Count = 1 Then
                  DriveStartClock = Selection.Offset(0, 4)
              Else: DriveStartClock = Sheets("Parsed Data").Cells(OutputRow - 1, 9)
              End If
              
              End Function
              Function DriveNum(ByRef Selection As Range)
              
              If Selection.Hyperlinks.Count = 1 Then
                  DriveNum = Selection
              Else: DriveNum = Sheets("Parsed Data").Cells(OutputRow - 1, 10)
              End If
              
              End Function
              Function PlayNum(ByRef Selection As Range)
              
              FGPos = FindString(Selection.Value, "field goal attempt ", 1)
              PuntPos = FindString(Selection.Value, "punt ", 1)
              PATPos = FindString(Selection.Value, "kick attempt ", 1)
              NegatedPos = FindString(Selection.Value, "NO PLAY", 1)
              DriveNumber = Sheets("Parsed Data").Cells(OutputRow - 1, 10)
              KOPos = FindString(Selection.Value, "kickoff ", 1)
              RunPos = Application.WorksheetFunction.Min(FindString(Selection.Value, "rush for ", 1), _
                           FindString(Selection.Value, "rush over ", 1), _
                           FindString(Selection.Value, "rush quarterback ", 1), _
                           FindString(Selection.Value, "rush option ", 1), _
                           FindString(Selection.Value, "rush up ", 1))
              PassPos = FindString(Selection.Value, "pass ", 1)
              
              x = 2
              
              End Function
              Everything in life is an approximation.

              http://twitter.com/CougarStats

              Comment


              • #8
                Originally posted by YOhio View Post
                Thanks man! You're like the Johannes Gutenberg of Perl scripts. For so long we, the ignorant masses, have been unable to do geeky statistical analysis. But you've unlocked the door for us. For this I'm certain you'll face scorn amongst your peers. Notice that Lebowski has kept silent.
                I am sure JL is fuming ... I have thrown open the curtain that divides the haves and the have nots. It is also good to see Indy help me with the curtain.

                Comment


                • #9
                  I've also created Excel VBA programs to download 110,000+ results that involve one or more current 1-A football team and piped that into an Access database and I've wrote another VBA program to get the complete game by game statistical totals for each team from 2000 to 2008.

                  I'm working on a couple of other things, but I haven't put much time into them in quite awhile.
                  Everything in life is an approximation.

                  http://twitter.com/CougarStats

                  Comment


                  • #10
                    Originally posted by Indy Coug View Post
                    I've also created Excel VBA programs to download 110,000+ results that involve one or more current 1-A football team and piped that into an Access database and I've wrote another VBA program to get the complete game by game statistical totals for each team from 2000 to 2008.

                    I'm working on a couple of other things, but I haven't put much time into them in quite awhile.
                    Indy, that is true dedication to the geek craft. It really does put my efforts to shame ... although you do get style point deductions for using VBA.

                    Comment


                    • #11
                      Originally posted by pelagius View Post
                      Indy, that is true dedication to the geek craft. It really does put my efforts to shame ... although you do get style point deductions for using VBA.
                      I'm a self-taught programmer (although I had a couple of introductory programming courses at BYU) and I had considerable work experience with Excel and developing VBA macros, so it was a natural jumping off point.

                      I don't have a freaking clue how to read PERL, let alone program in it, nor do I know any of the state of the art modern languages.

                      I did have fun a long time ago dabbling in APL programming, but the "fun" came from the monumental challenge of taking existing code, understand what it was doing and then modify it for additional functionality.
                      Everything in life is an approximation.

                      http://twitter.com/CougarStats

                      Comment


                      • #12
                        Originally posted by Indy Coug View Post
                        I don't have a freaking clue how to read PERL, let alone program in it, nor do I know any of the state of the art modern languages.
                        perl doesn't get you much style points these days either (more like street cred) ... It is an ugly language (but powerful and you can get stuff done quickly) ... python gets you style points (maybe Ruby too).
                        Last edited by pelagius; 08-19-2009, 07:36 PM.

                        Comment


                        • #13
                          Originally posted by pelagius View Post
                          perl doesn't get you much style points these days either (more like street cred) ... It is an ugly language (but powerful and you can get stuff done quickly) ... python gets you style points (maybe Ruby too).
                          Geek cred?

                          Perl definitely trumps anything with "VB" in its name.

                          Comment


                          • #14
                            Originally posted by beefytee View Post
                            Geek cred?
                            exactly

                            Comment


                            • #15
                              I do all my programming in COBOL.

                              Comment

                              Working...
                              X