Recent Posts

Uncategorized

How to get punched in the face by Go’s Standard Library (and arguably my own stupidity)

Posted on June 23, 2014

Ever have one of those days? A “I forgot to add a semi colon to terminate a line” type of day? Yeah, me too. And today was one of them.

Many developers, when allowing user uploaded data, tend to trust the web browser’s headers a little too much. I prefer to use MIME Type sniffing on the actual binary stream. This gives me a greater level of assurance that I’m not allowing a user to upload a different type of file than the one I desire. Say an executable binary with which to do my application damage.

I’ve been using using the net/http/sniff/DetectContentType function in the Go standard library to do this type of detection on images.

func ValidateImageType(b []byte) (string) {
  m := http.DetectContentType(b)
  switch m { 
  case "image/jpeg":
    return "jpg"
  case "image/png":
    return "png"
  case "image/gif":
    return "gif"
  }
  return ""
}

It works wonderfully.

Today I figured I’d implement mp4 detection so as to detect videos. I snagged a video file off of my Note 2 and created a quick test.

func ValidateVideoType(b []byte) (string) {
  m := http.DetectContentType(b)
  switch m { 
  case "video/mp4":
    return "mp4"
  }
  return ""
}

This didn’t work and I headed over to the documentation. After reading the function’s signature, I was pretty sure I was using it correctly. Suspecting that my mp4 was corrupt I went and opened up the file in a hex editor.

mp4_hexadecimal

Comparing it to the living specification, it appeared correct. But just in case the file was in fact corrupt, I went out and found a test mp4 file from Apple.com. Unfortunately this file also failed.

Knowing that I was using the function correctly, and that the file was intact, I headed over to review the sniff.go source file.

package main
 
import (
  "encoding/binary"
  "bytes"
  "log"
  "os"
  "io"
  "net/http"
)
 
func main() {
  var total int
  var ext string
  buf := make([]byte, 512)
 
  fi, err := os.Open("./sample_mpeg4.mp4")
  if err != nil {
    panic(err)
  }
  defer fi.Close()
 
  for {
    n, err := fi.Read(buf)
    total = total + n
    if n == 0 {
      break
    }
    if err == io.EOF {
      break
    }
    if err != nil {
      break
    }
 
    if n == 512 {
 
      data := buf[:n]
      if len(data) < 8 {
        break
      }
 
      boxSize := int(binary.BigEndian.Uint32(data[:4]))
      if boxSize%4 != 0 {
        break
      }
      if len(data) < boxSize {
        break
      }
      if !bytes.Equal(data[4:8], []byte("ftyp")) {
        break
      }
 
      for st := 8; st < boxSize; st+=4 {
        if st == 12 {
          continue
        }
        seg := string(data[st : st+3])
        switch seg {
          case"mp4", "iso", "M4V", "M4P", "M4B":
            ext = "mp4"
        }
        if len(ext) > 1 {
          break
        }
      }
 
      m := http.DetectContentType(buf)
      log.Println("LOCAL: ", ext)
      log.Println("SOURCE: ", m)
    }
  }
}

Bizarrely, the local version found the desired file type. Wuuuuuuuu?!?

I wondered if it was my OSX environment so I spun up a linux box, downloaded the Go source, and dove into the source code.

Making sure I was at least starting off in the right direction, I ran the bash script go/src/all.bash and all the tests completed successfully. I went to the sniff_test.go file and found this test dataset commented out.

        //{"MP4 video", []byte("\x00\x00\x00\x18ftypmp42\x00\x00\x00\x00mp42isom<\x06t\xbfmdat"), "video/mp4"},

Hmmmmm, that looked interesting. So I uncommented the code and ran the tests again. THEY FAILED! It wasn’t what I wanted, but at this point I’ll take anything.

Knowing I was close I did a recursive grep for the function signature in question to see what was using it.

backwardselvis@lts-linux:~/go/src$ grep -nr "mp4Sig" ./
./pkg/net/http/sniff.go:102:	//mp4Sig(0),
./pkg/net/http/sniff.go:164:type mp4Sig int
./pkg/net/http/sniff.go:166:func (mp4Sig) match(data []byte, firstNonWS int) string {
backwardselvis@lts-linux:~/go/src$

And immediately, my face struck my palm and my head hit my keyboard.

I opened up sniff.go. The EXACT SAME FILE that I’d been staring at all day and uncommented line 102. Saved, quit, and upon running the tests again they all passed successfully.

So there you have it. An unimplemented feature and traversing old VC subtrees leads to a bad day. Make sure you actually review the source you are working on.

:peace

You Should Support Clintmichigan’s Album “Coeur d’Alene”

My friend Clint Asay (and relative through marriage) has released his new album titled “Couer d’alene” and I’ve been having a great time listening to it. The album is awesome and I’ve included several tracks (the title track and the reprise are beautiful) into my programming playlists to keep me wee fingers coding during the long stretches.

A little background:

Clint scraped together the cash to record this, then turned around and self mixed and released the album. That is amazing! As someone who also creates through writing and music, I have to give anyone who completes something a giant high-five. That stuff is hard. And don’t get me started about releasing something personal like Clint has done. You are exposing your heart and mind to the internet and the world, which tends to be an extremely cruel mistress.

I’ve linked to his music, please stop by and support him when you get a second.

http://clintmichigan.bandcamp.com/

-peace
jared

Oracle 11g Enterprise Manager Error: Invalid Connection Pool. ERROR = ORA-28000: the account is locked

Problem:

You are trying to access the Oracle Enterprise Manager and are getting the following error.

Error:

Invalid Connection Pool. ERROR = ORA-28000: the account is locked

Explanation:

In my instance, the password for the SYSMAN database user was different than the emctl SYSMAN password. Every refresh I could see the errors messages indicating it was trying to connect, and then the account would become locked as the connection failed.

1) you need to track down the sysman/log directory

cd /u01/app/oracle/product/11.2.0/dbhome_1/example.com_orcl/sysman/log

2) tail the following log file

tail -f emoms.log

3) refresh your web browser and you should see the following error message pop up

Invalid Connection Pool. ERROR = ORA-28000: the account is locked

4) as the oracle user, login as sysdba

sqlplus / as sysdba

5) unlock the account

alter user sysman account unlock;

6) set a new password (don’t do this on production unless you have to)

alter user sysman identified by 'YOUR_NEW_PASSWORD';

7) now go to your command prompt and type

emctl setpasswd dbconsole

8) when it asks for your password, enter your new password

Please enter new repository password: YOUR_NEW_PASSWORD

9) close your web browser, open it and head to your em link

example.com:1160/em/console

10) enter following

User: sysman
Password: YOUR_NEW_PASSWORD

Howto: Resolve Centos 6 and Windows Share Error – CIFS VFS: cifs_mount failed w/return code = -22

Posted on August 6, 2012

When trying to mount a windows fileshare on centos 6.

[jared@localhost ~]$ sudo mount -t cifs //192.168.1.101/share /tmp/share
mount: wrong fs type, bad option, bad superblock on //192.168.1.101/Corpshare,
       missing codepage or helper program, or other error
       (for several filesystems (e.g. nfs, cifs) you might
       need a /sbin/mount.<type> helper program)
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

Tail dmesg and you will find the following error.

[jared@localhost ~]$ sudo dmesg | tail
Slow work thread pool: Ready
CIFS VFS: cifs_mount failed w/return code = -22

Install cifs-utils

[jared@localhost ~]$ sudo yum install cifs-utils

Try mounting it again

[jared@localhost ~]$ sudo mount -t cifs -o username=YOUR_USERNAME,password=YOUR_PASSWORD //192.168.1.101/share /tmp/share 
mount.cifs kernel mount options: ip=192.168.1.101,unc=\\192.168.1.101\share,,ver=1,user=YOUR_USERNAME,pass=********
[jared@localhost ~]$ cd /tmp/share
Older Posts