Symantec Anti-Virus Issues at CSU: It Is What It Is by Zube (zube@stat.colostate.edu) Created: Apr 17, 2008 Updated: Apr 16, 2012 http://www.stat.colostate.edu/~zube/sav.txt Introduction ------------ Symantec Anti-Virus Corporate Edition has been the chosen AV product at CSU for many years. I believed that it was no longer the best choice for the computers I manage and wished to switch to Avira AntiVir. This choice was blocked by the central IT support people (ACNS, Academic Computing and Networking Services). I must now use SAV (now SEP or Symantec Endpoint Protection) against my better judgment and without my consent, so I thought it would be interesting to document my reasons for wanting to switch. This document is also an experiment: "When there's something we can't say, it's often because some group doesn't want us to. The prohibition will be strongest when the group is nervous." This may also be appropriate: "University politics are vicious precisely because the stakes are so small." Prologue -------- Symantec Anti-Virus (originally Norton Anti-Virus) Corporate Edition replaced F-Prot as the campus choice sometime at or around SAV 7.03. The 7.x versions of Symantec were more than adequate and seemed to run fine. The 8.x versions were slower, but still good enough. The campus skipped the 9.x versions and went to 10.x, which turned most low-end machines to mush; I wanted to move away in 2006, but the campus renewed its license early making any such move impractical. Today, in April 2008, I am faced with SEP. My initial tests show that it is even slower than SAV and my gut tells me that the decision to use SEP will end in tears. However, I'm often wrong and sincerely hope I am wrong this time. Indeed, I may be wrong. If everyone sticks to 10.1.x and no security issues are found in 10.1.x for two years, the campus will be no worse off than it is now. Or if the Hot, New SEP Suite is deployed without the Hot part or the New part or the Suite part, there may be no tears, only the same low-level pain that has been around since 10.x was introduced. For the record, here are some problems I've experienced with SAV since 2004. Technical Problems related to ACNS ---------------------------------- 1) In 2005/2006, virus definitions were frequently noted as "out of date" on clients that were set to pull from the ACNS server. In my notes, the following dates are listed: Sep 28, 2005 Nov 4, 2005 Dec 2, 2005 Dec 23, 2005 Apr 3, 2006 May 19, 2006 I started writing these down only after I had seen the problem for a while. I eventually set up LiveUpdate to pull from Symantec rather than from ACNS to insure that the clients always had up-to-date definitions. On July 24, 2008, ACNS announced that the virus definitions had been out of date since July 17, 2008, possibly due to server configuration changes. 2) The central Symantec server was upgraded by ACNS without warning sometime between the end of July and the beginning of August 2005, causing the latest LiveUpdate (at the time, 2.7.34) to fail. I then had to visit each machine, remove the latest LiveUpdate and install an older one (2.6.14) to fix the issue. 3) When a security issue with UPX files in SAV 8.x was found in February of 2005, I wanted to know if it also affected 7.x. I found the answer, not from ACNS, but from asking very sweetly in the Symantec forums. I was also told by ACNS *in February 2005* that support for 7.x was to expire in *in March 2005*. This timetable had been published by Symantec at least two years previous but ACNS failed to tell me (and possibly everyone) about it. 4) On Apr 24, 2006, I was kindly offered a SAV 10.1 cdrom and was told "Barring any major concerns with this release, our plan is to recommend this version to the campus for summer/fall upgrades." I installed SAV 10.1, but I also installed two patches to 10.1 (available on the Symantec web site) at an enormous expense of time. ACNS never mentioned either patch when I was given the cdrom. I was glad I did the extra work: http://www.symantec.com/avcenter/security/Content/2006.05.25.html ACNS then went with SAV 10.1.4 in the fall. Logistical Problems related to ACNS ----------------------------------- 1) On June 22, 2004, a message regarding SAV was sent out to the subnet-managers list. It contained this: "Please take your payment of $12.41 for each computer that will have Norton AntiVirus installed to the Software Cellar between July 1 - July 16, 2004. Media (CD) with Symantec AntiVirus 8.1 is available for $5 each." On July 9, 2004 I went to the Software Cellar to renew my licenses. I also asked for a copy of the media because of the above announcement. I was given a copy of 7.61 which I promptly returned. When I asked about 8.1, I got a shrug. I then forgot about it and thought there would be great fanfare when 8.1 was available, but if there was, I missed it. In February 2005, I was told that SAV 8.1 had been given out since the summer, specifically the second week in August. News to me. At any rate, it would have been much too late in the summer to do anything for the fall semester. 2) The nth Symantec license costs $X, but the n+1th Symantec license costs $X + $Y, where n is the number of licenses originally "ordered" by the Software Cellar. For example, I purchased an SAV license on 10/04/07. It cost me $13.90. If you examine the Software Cellar cost page, you'll find that the current cost is $21.20. In past years, this has been explained to me as "We ran out of licenses, so we had to order more, so the price went up," which doesn't make any sense (to me anyway). On all the license bits I manage, the n+1th license cost is no more than the nth license cost and it is frequently less. 3) In March of 2006, ACNS renewed the University's Symantec license early. A message was sent out which read: *** The implications for departments would be they would need to pay for the Symantec Antivirus renewal by April 15 instead of a June time frame. Because our renewal comes due in June, departments hopefully already have money in this year's budget for the AV renewal. *** Two years earlier, this message was sent out (same one as above): *** Please take your payment of $12.41 for each computer that will have Norton AntiVirus installed to the Software Cellar between July 1 - July 16, 2004. Media (CD) with Symantec AntiVirus 8.1 is available for $5 each." *** We had always paid for SAV in July, the first month of a new fiscal year. ACNS does not know its own payment history. 4) Here's an interesting MS KB article that details which directories should be excluded from virus scanning: http://support.microsoft.com/default.aspx?kbid=822158 It contains this sentence: "If you scan these files, serious performance problems may occur because of file locking." One might expect that the Anti-Virus experts would know about this and pass it along. 5) Email exchanges with ACNS regarding my desire to change to AntiVir were ... interesting. After much ping-pong wherein they again and again refused to answer direct questions, I was told flatly that I must run the current versions of the standard Anti-Virus program. Later, it was my honor also to be told that I was doing this out of preference and not necessity. If my sources are correct, the CSU Business School does not use SAV/SEP. I'm sure that as soon as the "sun wakes up in the west and lays its head down in the east," we'll see ACNS enforce this rule uniformly. Double-standards are *extremely* useful tools. Technical Problems related to Symantec -------------------------------------- 1) The 10.x series is slow, due in part to the Startup Scan that accompanies every login by default. Sometimes it takes minutes between login and the time one is able to start working. The baddie in question (doscan.exe) is unkillable once it starts. The scan can be disabled, but it can be disabled only on a per-user basis. Also, by default, the update of virus definitions triggers yet another scan. 2) When a security hole was discovered in 10.1.4 and previous versions, the only remedy was to upgrade to 10.1.6. There was no patch to fix the hole in any previous version. After 10.1.6 was installed, a patch for it was released (MP1). That patch caused the following problem: The first user to log in on a machine is fine. If that user logs out and anyone else (including the first user) logs back in, both port 25 and 110 are blocked. One cannot receive mail via insecure pop nor can one send mail out. A reboot fixes the problem for the first login again. This has forced anyone using a mail client to reboot after each login and it has caused me many wasted hours explaining the problem to faculty, staff and students. Symantec has never fixed the issue. The only paths around the problem are to not install the patch, or to upgrade to SAV 10.1.7 or SEP. 3) Symantec has peer-to-peer forums, but seems blissfully unaware of any content on them. A known problem with SEP MR1 with older versions of Cygwin was mentioned in January 2008 on the forums, but in March 2008 Symantec tech support had no reference for the problem since no one reported it directly to Symantec. 4) A recent LiveUpdate (3.1 or 3.2) installed several DLLs but the permissions were set too restrictively. The Administrator account was fine, but non-Administrative users were treated to SAV crashing on login. The latest LiveUpdate for corporate users (3.3.0.61) does not have the problem, but it took Symantec several months to fix the issue. 5) The "latest" LiveUpdate seems to disappear randomly. For example, On August 20, 2008, I checked the LiveUpdate page and version 3.3.x was missing, although that version was still mentioned in the version 3.4 blurb. Version 3.2 was there in its stead. The next time I checked, version 3.3.0.71 was there. 6) With defs dated Nov 21, 2007 rev 2, LiveUpdate stopped pulling and installing new defs from the Symantec site. It turns out that this could be fixed by a reboot or by stopping and starting the Symantec Antivirus service. Personal observations --------------------- 1) SEP is slower than SAV. When I have to install it, I reach for SAV 10.1.x (now 10.1.9) instead. 2) SEP may be the single greatest program ever written, but one thing is clear: people are not breaking down doors to get to it. Combing over the many newsgroups and mailing lists that I read, I find that SEP is generally panned. Admittedly, there are some people who are deploying SEP either because they are forced to by management (hmm, that sounds familiar) or because they think it is "good enough." However, I've yet to find anyone who is leaving a competing Anti-Virus product *for* SEP. I find that most curious. SEP issues ---------- 1) There is an issue with older versions of Cygwin and SEP. The error one sees is: c:\temp> rm file.exe 17[unknown (0x700)]? 2184 init_cheap: Couldn't reserve space for cygwin's heap, Win32 error 487 c:\cygwin\bin\rm.exe *** AllocationBase 0x0, BaseAddress 0x61600000, RegionSize 0x150000, State 0x100000 If one upgrades to a recent version of Cygwin, the problem does not occur. If one does not install the Intrusion bits of SEP, the problem does not occur. 2) This thread: https://forums.symantec.com/syment/board/message?board.id=endpoint_protection11&thread.id=2221 (one line) is interesting. A user noted that the memory footprint for SEP on Windows 2000 was larger than on XP and that no workaround could be found except to upgrade to XP. Here was a reply from a "Symantec Endpoint Moderator" Paul Murgatroyd: "Windows 2000 does not support trimming the working set, so we have to allocate more memory to each component as we can't dynamically decrease the component usage. With XP and beyond, because we can trim the working set, we can decrease the amount of memory used in the working set." This seems to contradict the "SEP is lean and mean" messages coming from ACNS. Or perhaps Windows 2000, an OS in its declining years but still supported by Microsoft, simply doesn't count. More to come, certainly. Or see for yourself at the SEP forums: http://www.symantec.com/connect/security/forums/endpoint-protection Prediction ---------- In 2010, ACNS will drop Symantec like a hot potato and run into the arms of Microsoft, specifically MS Forefront; all the reasons for staying with Symantec, all of the justification as to why SEP was chosen will be dropped down the Memory Hole. Updates: [Aug 2008] After much ado, ACNS went with an 11-month license for Symantec for 2008-2009. The price was generally in line with previous years ($7.25), so kudos for that. Sadly, they are "moving forward" with SEP and have already put it out for download. If the uptake of SEP is large, I think things will get very interesting, but as before, I'm often wrong, so we'll see. [Jun 2009] I'm happy to admit that my prediction was wrong. ACNS took 8 months (!) of meetings to decide to keep Symantec for another three years. This is good news, if only that I don't have to run around and install Forefront after running around and installing SAV 10.1.8. Since I can't resist at least one dig, here are some of the critia (according to ACNS) that the choice was based on: product features anti-virus philosophies and tactics enterprise management systems corporate stability endpoint resource utilization/performance impact tech support number of OS platforms covered effectiveness of detection/removal A fine list. Note that both previous performance and cost are not listed. [Apr 2012] Along with no one actually migrating to SEP, it turns out that Symantec completely pulled out of the av-comparatives testing for 2012: http://www.av-comparatives.org/forum/index.php?page=Thread&threadID=1060 Every other major vendor is there, including Microsoft.